-- Logs begin at Fri 2022-09-09 01:00:30 UTC, end at Fri 2022-09-09 02:14:58 UTC. -- Sep 09 01:34:02.915045 dv-cinder-852210-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 01:34:02.923764 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 01:34:02 2022] *** Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: nodename: dv-cinder-852210-12-iscsi Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: machine: x86_64 Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: clock source: unix Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: pcre jit disabled Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: detected number of CPU cores: 4 Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: current working directory: / Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: detected binary path: /usr/bin/uwsgi-core Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: your processes number limit is 47680 Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: your memory page size is 4096 bytes Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: detected max file descriptor number: 2048 Sep 09 01:34:02.925816 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: lock engine: pthread robust mutexes Sep 09 01:34:02.926530 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: thunder lock: enabled Sep 09 01:34:02.926530 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 01:34:02.926530 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 01:34:02.965845 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: Python main interpreter initialized at 0x55af367cd9c0 Sep 09 01:34:02.965845 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: python threads support enabled Sep 09 01:34:02.965845 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: your server socket listen backlog is limited to 100 connections Sep 09 01:34:02.965845 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: your mercy for graceful operations on workers is 90 seconds Sep 09 01:34:02.966180 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 01:34:02.966258 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: *** Operational MODE: preforking *** Sep 09 01:34:02.966524 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: *** uWSGI is running in multiple interpreter mode *** Sep 09 01:34:02.966524 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: spawned uWSGI master process (pid: 87870) Sep 09 01:34:02.966712 dv-cinder-852210-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 01:34:02.968856 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: spawned uWSGI worker 1 (pid: 87883, cores: 1) Sep 09 01:34:02.968856 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: spawned uWSGI worker 2 (pid: 87884, cores: 1) Sep 09 01:34:02.968856 dv-cinder-852210-12-iscsi devstack@keystone.service[87870]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 01:34:07.227560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 01:34:07.227560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: from cryptography.utils import int_from_bytes Sep 09 01:34:07.228305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 01:34:07.228305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: from cryptography.utils import int_from_bytes Sep 09 01:34:07.228716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 01:34:07.228716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: from cryptography.utils import int_from_bytes Sep 09 01:34:07.228825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 01:34:07.228825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: from cryptography.utils import int_from_bytes Sep 09 01:34:07.278153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 01:34:07.278313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 01:34:07.278401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 01:34:07.278489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 01:34:07.278562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] command line args: [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 01:34:07.278676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] command line args: [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 01:34:07.278760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 01:34:07.278935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ================================================================================ {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 01:34:07.279021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 01:34:07.279097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ================================================================================ {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 01:34:07.279171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] admin_token = **** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.279349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] admin_token = **** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.279438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] config_dir = [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.279585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.279664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] config_dir = [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.279765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] config_source = [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.279843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.279943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] config_source = [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] debug = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] debug = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.280972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] list_limit = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] list_limit = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] log_config_append = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] log_config_append = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.281987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] log_dir = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] log_dir = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] log_file = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] log_file = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.282934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.283940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284807 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.284970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.285135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.285217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.285288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.285413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.285736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.285864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.285954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] publish_errors = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] publish_errors = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.286915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] standard_threads = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] standard_threads = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.287955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] use_journal = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] use_json = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] use_journal = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] use_stderr = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] use_json = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] use_syslog = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] use_stderr = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] use_syslog = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 01:34:07.288982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.289129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.289221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.289370 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.289449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.289608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.289691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.289848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.289931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.290081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.290165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.290323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.290405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.290554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.290637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.290790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.290873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.291960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.292044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.292190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.292268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.292931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.293020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.293203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.293283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.293454 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.293533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.293705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.293786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.293944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.294984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.295146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.295223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.295372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.295458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.295598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.295681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.295823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.295898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296749 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.296977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.297054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.297233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.297310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.297474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.297565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.297709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.297790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.297942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.298936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.299970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.300999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.301127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.301229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.301370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.301512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.301610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.301698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.301816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.301890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.302933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.303973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.304086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.304174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.304325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.304403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.304553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.304635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.304780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.304859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.305990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306757 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.306989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.307144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.307221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.307375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.307460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.307606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.307683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.307835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.307913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.308665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.308753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.308896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.308974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.309150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.309233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.309385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.309466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.309615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.309693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.309846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.309923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.310080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.310160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.310318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.310400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.310558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.310636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.310792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.310877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.311936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.312915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.313091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.313174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.313308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.313395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.313538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.313626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.313769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.313847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.314946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] auth.external = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] auth.external = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315757 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.315990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] auth.password = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.316129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] auth.token = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.316206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] auth.password = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.316353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] auth.token = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.316430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.316579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.316726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.316845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.316919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317759 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.317989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.318074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] credential.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.318223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] credential.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.318309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.318463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.318541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.318705 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.318793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.318941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.319962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.320115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.320202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.320352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.320431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.320603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.320681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.320849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.320935 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.321127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.321222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.321366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.321453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.321613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.321699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.321841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] federation.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.321925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.322072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.322152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] federation.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.322306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.322392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.322547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.322626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.322774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.322852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.323965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.324973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.325164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.325247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.325398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.325477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.325628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.325704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.325862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.325942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.326085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.326162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.326316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.326395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.326547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.326626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.326793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.326872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.327939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.328984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.329169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.329251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.329403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.329484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.329631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.329710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.329857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.329935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.330083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.330162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.330311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.330394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.330545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.330623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.330781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.330862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331101 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.331957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.332963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.333131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.333212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.333361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.333450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.333590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.333669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.333820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.333898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.334049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.334136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.334281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.334388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.334622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.334700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.334867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.334953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.335095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.335175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.335323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.335402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.335550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.335639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.335799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.335878 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.336944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.337029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.337207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.337288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.337439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.337517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.337681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.337771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.337922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.338919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.339069 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.339155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.339298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.339377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.339547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.339625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.339773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.339859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.340958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.341998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.342142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.342226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.342375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.342455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.342613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.342700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.342846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.342925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.343075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.343153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.343307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.343398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.343542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.343620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.343771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.343849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.344921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345173 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] resource.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] resource.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.345962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.346116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.346203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.346350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.346434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.346590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.346668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.346827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.346914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347370 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] role.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.347969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] role.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] role.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] role.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348745 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.348990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.349174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.349257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.349417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.349497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.349658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.349739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.349899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.349977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.350138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.350216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.350366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.350446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.350599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.350679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.350840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.350919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.351083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.351162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.351321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.351402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.351562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.351708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.351942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.352984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.353157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.353252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.353402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.353480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.353646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.353724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.353881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.353968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.354110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.354190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.354338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.354416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.354563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.354648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.354791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.354869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.355562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.355643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.355805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.355891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] token.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.356948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] token.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.357999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.358153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.358231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.358386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.358464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.358622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.358709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.358860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.358938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.359088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.359166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.359321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.359408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.359550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.359630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.359780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.359857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.360987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.connection = **** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.361083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.361246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.361330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.connection = **** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.361486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.361566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.361717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.361797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.361950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.362946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.363099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.363178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.363324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.363404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.363560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.363639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.363787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.363865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364101 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.364938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.365016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.365189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.365269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.365354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87884) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 01:34:07.365429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 01:34:07.365578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87883) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 01:34:07.394792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.395022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.395248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.395333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.395480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.395708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.403584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.404071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.431742 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.431969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.432890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.433201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.485620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.485914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.533877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.534106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.534623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.534860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.689731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.689952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.690173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.690261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.690391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.690498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.690608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.690691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.690828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.690909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.691051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.691267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.713488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.713717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.713802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.713938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.714111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.714197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.714271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.714901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.714997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.715107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.715240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.715326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.715414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.715546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.726365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.727044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.739341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.739453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.739533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.739619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.739725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.739811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.739944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.740028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.740160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.740253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.740373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.740457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.740587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87884) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.740670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87883) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 01:34:07.758338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.758546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 01:34:07.758785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.758898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.758982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 01:34:07.759097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 01:34:07.759320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.759404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.759518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 01:34:07.759622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.759790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.759881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.760055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.760140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.760314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.760572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.770259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.770543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.770732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.770815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.771012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.771094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.771284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.771568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.778074 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.778540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.780960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.781274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.781429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.781566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.781716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.781855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.782002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.782145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.782290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.782410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.782570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.782683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.782829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.782961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.783099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.783235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.783372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.783511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.783643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.783780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.783916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.784183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.806387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.806730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.807845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.808649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.808737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.808955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.813403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.813609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.813692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.813894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.813976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.814190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.820259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.820457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.820542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.820743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.826672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.826935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.827021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.827232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.827315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.827523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.827613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.827805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.827886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.828087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.828169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.828365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.828447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.828637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.828717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.828907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.844025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.844308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.844391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.844663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.848119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.848527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.851336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.851636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.851763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.852063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.855830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.856289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.858962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.859466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.862701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.863045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.863394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.863682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.863968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.864252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.864332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.864624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.864911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.865228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.865498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.865790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.881322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.881615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.881883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.882147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.885587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.885880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.886151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.886471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.888981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.890528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.892127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.892306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.893301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.896274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.896881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.896881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.896881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.896881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.896881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.897237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.897237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.899409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.899699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.899994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.900278 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.900673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.907824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.908110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.908392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.908669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.909048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.909359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.909639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.912341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.912626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.912910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.913219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.913603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.913897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.914179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.925796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.928675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.930101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.931560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.932995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.933133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.934592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.934875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.935269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.935931 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.937434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.938896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.939181 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.939585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.941668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.943118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.945955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.946057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.946385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.946673 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.946947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.947419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.950840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.951238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.951529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.951813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.955169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.955460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.955739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.959593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.959892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.960177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.962587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87884) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.962871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.963152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.963431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.963706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.963989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.964262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.965135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.965411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.965691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.965969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.966232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.966497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.966790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87884) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.967642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87883) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 01:34:07.967927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.968211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.968493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.968772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.969048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.969365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.969660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.969942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.970224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.970506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.970774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.971041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:07.971478 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87883) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 01:34:08.000513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55af367cd9c0 pid: 87884 (default app) Sep 09 01:34:08.003743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217728-3130-4c02-86e9-b212be953279 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:08.004027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217728-3130-4c02-86e9-b212be953279 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:08.004295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87217728-3130-4c02-86e9-b212be953279 None None] PATH_INFO: `/v3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:08.005566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55af367cd9c0 pid: 87883 (default app) Sep 09 01:34:08.008671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Fri Sep 9 01:34:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:14.751731 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc03f-b0e5-40db-938d-73dd94b5ff3d None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:14.752249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc03f-b0e5-40db-938d-73dd94b5ff3d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:14.752249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc03f-b0e5-40db-938d-73dd94b5ff3d None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:14.756783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:14.762043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:14.762612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:14.762612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:14.832536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: INFO dbcounter [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] Registered counter for database keystone Sep 09 01:34:14.851181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG oslo_db.sqlalchemy.engines [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87884) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 01:34:14.857910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writer thread running {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 01:34:15.039947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG passlib.handlers.bcrypt [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87884) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 01:34:15.040245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG passlib.handlers.bcrypt [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87884) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 01:34:15.098384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:15.099998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:15.228928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 01:34:15.647824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc03f-b0e5-40db-938d-73dd94b5ff3d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:15.716961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: INFO dbcounter [None req-43edc03f-b0e5-40db-938d-73dd94b5ff3d None None] Registered counter for database keystone Sep 09 01:34:15.738592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG oslo_db.sqlalchemy.engines [None req-43edc03f-b0e5-40db-938d-73dd94b5ff3d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87883) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 01:34:15.744825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writer thread running {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 01:34:15.862826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc03f-b0e5-40db-938d-73dd94b5ff3d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:15.867162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353469f9-a92a-4169-8ec2-f32d8c6f319a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:15.868046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353469f9-a92a-4169-8ec2-f32d8c6f319a admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:15.868331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353469f9-a92a-4169-8ec2-f32d8c6f319a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:15.868601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353469f9-a92a-4169-8ec2-f32d8c6f319a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:15.877310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353469f9-a92a-4169-8ec2-f32d8c6f319a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:15.929686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WARNING keystone.common.rbac_enforcer.enforcer [None req-353469f9-a92a-4169-8ec2-f32d8c6f319a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 01:34:15.929982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353469f9-a92a-4169-8ec2-f32d8c6f319a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:15.939661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Fri Sep 9 01:34:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:34:15.945559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:15.962091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970ea8c-2e56-4cf1-a9d1-b47485ab4422 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:15.967048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f455d0-9a5d-4698-a9a5-3b5a78e96b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:15.968156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f455d0-9a5d-4698-a9a5-3b5a78e96b5a admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:15.968437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f455d0-9a5d-4698-a9a5-3b5a78e96b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:15.968989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f455d0-9a5d-4698-a9a5-3b5a78e96b5a admin admin] PATH_INFO: `/v3/projects/af9d764870ec46c58ab37a817dc46423` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:15.970622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f455d0-9a5d-4698-a9a5-3b5a78e96b5a admin admin] RBAC: Authorizing `identity:get_project(project_id=af9d764870ec46c58ab37a817dc46423)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:16.023748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e2f455d0-9a5d-4698-a9a5-3b5a78e96b5a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 01:34:16.024091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f455d0-9a5d-4698-a9a5-3b5a78e96b5a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:16.029107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Fri Sep 9 01:34:15 2022] GET /identity/v3/projects/af9d764870ec46c58ab37a817dc46423 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:18.130886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e861e-1955-4374-88cb-a01323df336b None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:18.131439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e861e-1955-4374-88cb-a01323df336b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:18.131439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e861e-1955-4374-88cb-a01323df336b None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:18.135493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:34:18.141421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052c5b-6e9e-4530-866c-c672bec42790 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:18.141805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052c5b-6e9e-4530-866c-c672bec42790 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:18.141971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37052c5b-6e9e-4530-866c-c672bec42790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:18.161620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-37052c5b-6e9e-4530-866c-c672bec42790 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:18.163085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-37052c5b-6e9e-4530-866c-c672bec42790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:18.170877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 01:34:18.441633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e861e-1955-4374-88cb-a01323df336b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:18.454220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e861e-1955-4374-88cb-a01323df336b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:18.458472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a1b7b-222b-441c-b6dc-16cace2e3927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:18.459068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1b7b-222b-441c-b6dc-16cace2e3927 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:18.459351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1b7b-222b-441c-b6dc-16cace2e3927 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:18.459625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1b7b-222b-441c-b6dc-16cace2e3927 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:18.467808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a1b7b-222b-441c-b6dc-16cace2e3927 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:18.469602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a1b7b-222b-441c-b6dc-16cace2e3927 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:18.478271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Fri Sep 9 01:34:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:34:18.483992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37052c5b-6e9e-4530-866c-c672bec42790 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:18.493148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37052c5b-6e9e-4530-866c-c672bec42790 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:18.498103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f7ec8-a942-47b8-a515-14d31e9b182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:18.498726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f7ec8-a942-47b8-a515-14d31e9b182b admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:18.499006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f7ec8-a942-47b8-a515-14d31e9b182b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:18.499281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f7ec8-a942-47b8-a515-14d31e9b182b admin admin] PATH_INFO: `/v3/users/2a2e93b317db4de6850826f625ccc3db` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:18.500585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f7ec8-a942-47b8-a515-14d31e9b182b admin admin] RBAC: Authorizing `identity:get_user(user_id=2a2e93b317db4de6850826f625ccc3db)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:18.502431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f7ec8-a942-47b8-a515-14d31e9b182b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:18.503954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Fri Sep 9 01:34:18 2022] GET /identity/v3/users/2a2e93b317db4de6850826f625ccc3db => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:20.734295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19471c-83fa-4733-9fc1-30c6a179aee2 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:20.734852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19471c-83fa-4733-9fc1-30c6a179aee2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:20.734852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19471c-83fa-4733-9fc1-30c6a179aee2 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:20.736169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:34:20.741544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421160-b52c-4fc2-a514-8c0759c7633d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:20.741971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421160-b52c-4fc2-a514-8c0759c7633d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:20.742219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d421160-b52c-4fc2-a514-8c0759c7633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:20.752710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c3b20-cdc0-4613-adce-1b96f827b423 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:20.752993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c3b20-cdc0-4613-adce-1b96f827b423 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:20.753302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c3b20-cdc0-4613-adce-1b96f827b423 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:20.754840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:34:20.759992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:20.760276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:20.760549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:20.760821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-2d421160-b52c-4fc2-a514-8c0759c7633d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:20.762210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2d421160-b52c-4fc2-a514-8c0759c7633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:20.802519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG passlib.handlers.bcrypt [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87883) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 01:34:20.802904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG passlib.handlers.bcrypt [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87883) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 01:34:20.842511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:20.851942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:20.853545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:20.957184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:21.116772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d421160-b52c-4fc2-a514-8c0759c7633d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:21.141937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:21.221706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:21.222245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d421160-b52c-4fc2-a514-8c0759c7633d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:21.257735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94098e5d-2dba-4da3-ad96-44bb1e84b88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:21.258367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94098e5d-2dba-4da3-ad96-44bb1e84b88a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:21.258650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94098e5d-2dba-4da3-ad96-44bb1e84b88a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:21.258928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94098e5d-2dba-4da3-ad96-44bb1e84b88a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:21.270016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94098e5d-2dba-4da3-ad96-44bb1e84b88a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-94098e5d-2dba-4da3-ad96-44bb1e84b88a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:21.279511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:21.280422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:21.281551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:34:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:21.286383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94098e5d-2dba-4da3-ad96-44bb1e84b88a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:21.320907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d47e5-bba1-4630-a06a-6ad329ba000d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:21.323445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:21.358318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e08005-dfd3-4071-b950-e52be41857bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:21.359167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08005-dfd3-4071-b950-e52be41857bf None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:21.359444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08005-dfd3-4071-b950-e52be41857bf None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:21.359715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e08005-dfd3-4071-b950-e52be41857bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:21.367447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:21.387734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94098e5d-2dba-4da3-ad96-44bb1e84b88a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:21.421663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193d67a-0681-45d6-bced-240125702560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:21.422259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193d67a-0681-45d6-bced-240125702560 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:21.422541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193d67a-0681-45d6-bced-240125702560 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:21.423134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193d67a-0681-45d6-bced-240125702560 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:21.423793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193d67a-0681-45d6-bced-240125702560 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:21.424094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193d67a-0681-45d6-bced-240125702560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:21.425504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193d67a-0681-45d6-bced-240125702560 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:21.425946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e08005-dfd3-4071-b950-e52be41857bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:21.427443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e08005-dfd3-4071-b950-e52be41857bf None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:21.436843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:21.442479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193d67a-0681-45d6-bced-240125702560 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:21.522645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 01:34:21.528458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e08005-dfd3-4071-b950-e52be41857bf None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:21.531270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:21.541927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193d67a-0681-45d6-bced-240125702560 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:21.574896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f8bc0-5ebb-47bc-953e-2b47f408d4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:21.575520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8bc0-5ebb-47bc-953e-2b47f408d4df None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:21.575795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8bc0-5ebb-47bc-953e-2b47f408d4df None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:21.576063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f8bc0-5ebb-47bc-953e-2b47f408d4df None admin] PATH_INFO: `/v3/users/2a2e93b317db4de6850826f625ccc3db` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:21.577234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8bc0-5ebb-47bc-953e-2b47f408d4df None admin] RBAC: Authorizing `identity:get_user(user_id=2a2e93b317db4de6850826f625ccc3db)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:21.578670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f8bc0-5ebb-47bc-953e-2b47f408d4df None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:21.580411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:21 2022] GET /identity/v3/users/2a2e93b317db4de6850826f625ccc3db => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:21.585536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f8bc0-5ebb-47bc-953e-2b47f408d4df None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:21.600066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:21.625918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e08005-dfd3-4071-b950-e52be41857bf None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:21.659583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:21.660432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:21.660708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:21.660977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:21.679495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:21.680945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:21.682105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f8bc0-5ebb-47bc-953e-2b47f408d4df None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:34:21.699721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:21.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:34:21.701424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:34:21.701424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:34:21.701424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:21.702181 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:34:21.707789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:21.721218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e2005-2b6a-4e12-949a-bf19d4848ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:21.721864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e2005-2b6a-4e12-949a-bf19d4848ec8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:21.722144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e2005-2b6a-4e12-949a-bf19d4848ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:21.722417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8e2005-2b6a-4e12-949a-bf19d4848ec8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:21.723566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e2005-2b6a-4e12-949a-bf19d4848ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:21.725022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8e2005-2b6a-4e12-949a-bf19d4848ec8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:21.726770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:21.733428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e2005-2b6a-4e12-949a-bf19d4848ec8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:21.771145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:21.812687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:21.817132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83653910-4beb-4029-81d6-3cc941839e27 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:21.833495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8e2005-2b6a-4e12-949a-bf19d4848ec8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:21.849907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbefb6f-2d6d-4834-90d0-a0aeeb6e9f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:21.850510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbefb6f-2d6d-4834-90d0-a0aeeb6e9f15 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:21.850787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbefb6f-2d6d-4834-90d0-a0aeeb6e9f15 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:21.851058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbefb6f-2d6d-4834-90d0-a0aeeb6e9f15 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:21.851692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbefb6f-2d6d-4834-90d0-a0aeeb6e9f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:21.851994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbefb6f-2d6d-4834-90d0-a0aeeb6e9f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:21.853403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbefb6f-2d6d-4834-90d0-a0aeeb6e9f15 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:21.865342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91735743-888b-41d8-9d79-090a8b08fbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:21.865936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735743-888b-41d8-9d79-090a8b08fbd9 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:21.866214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735743-888b-41d8-9d79-090a8b08fbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:21.866482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91735743-888b-41d8-9d79-090a8b08fbd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:21.867191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91735743-888b-41d8-9d79-090a8b08fbd9 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.domain.id=default, user.id=2a2e93b317db4de6850826f625ccc3db) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:21.867500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91735743-888b-41d8-9d79-090a8b08fbd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:21.868906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91735743-888b-41d8-9d79-090a8b08fbd9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:21.869971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 10/20] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:21.876156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 11/21] 10.222.0.15 () {56 vars in 1322 bytes} [Fri Sep 9 01:34:21 2022] GET /identity/v3/role_assignments?user.id=2a2e93b317db4de6850826f625ccc3db&scope.domain.id=default&role.id=b131b61299154544a627315649090a49 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:24.627340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ebdf2-d8c6-41ce-beba-4c43b99c6f79 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:24.628347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ebdf2-d8c6-41ce-beba-4c43b99c6f79 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:24.628831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ebdf2-d8c6-41ce-beba-4c43b99c6f79 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:24.630658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 11/22] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:24.636635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4cdf5-b3e2-4de4-a2d5-eacfc111ef60 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:24.637503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4cdf5-b3e2-4de4-a2d5-eacfc111ef60 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:24.638093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4cdf5-b3e2-4de4-a2d5-eacfc111ef60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:24.661187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-38c4cdf5-b3e2-4de4-a2d5-eacfc111ef60 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:24.662659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-38c4cdf5-b3e2-4de4-a2d5-eacfc111ef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:24.709836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f4c4-30d9-45ce-8d8c-241ae35cc7b4 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:24.710479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f4c4-30d9-45ce-8d8c-241ae35cc7b4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:24.710977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f4c4-30d9-45ce-8d8c-241ae35cc7b4 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:24.712519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 12/23] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:34:24.718544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a05c12-3df1-4cfe-a565-7f4ada52c749 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:24.719073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a05c12-3df1-4cfe-a565-7f4ada52c749 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:24.719543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a05c12-3df1-4cfe-a565-7f4ada52c749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:24.749494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c1a05c12-3df1-4cfe-a565-7f4ada52c749 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:24.751066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c1a05c12-3df1-4cfe-a565-7f4ada52c749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:24.773710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 12/24] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:24.775396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec110c3-6e75-490f-80da-b0915559b868 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:24.777780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec110c3-6e75-490f-80da-b0915559b868 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:24.778966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec110c3-6e75-490f-80da-b0915559b868 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:24.780611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 13/25] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:24.786705 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe261-4acc-4394-9510-709180e3b65b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:24.787221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe261-4acc-4394-9510-709180e3b65b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:24.787712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe261-4acc-4394-9510-709180e3b65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:24.814792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-5e6fe261-4acc-4394-9510-709180e3b65b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:24.816396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5e6fe261-4acc-4394-9510-709180e3b65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:24.891048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 13/26] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:24.931706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 14/27] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:25.040870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69b4d8-a007-4b52-8684-3dbc88a88e85 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:25.041549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69b4d8-a007-4b52-8684-3dbc88a88e85 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:25.042023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69b4d8-a007-4b52-8684-3dbc88a88e85 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:25.043600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 14/28] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:25.050122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ca570-bf29-4f53-a8cd-729e0344a2ed None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:25.050410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ca570-bf29-4f53-a8cd-729e0344a2ed None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:25.050684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ca570-bf29-4f53-a8cd-729e0344a2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:25.061131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:25.078061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-365ca570-bf29-4f53-a8cd-729e0344a2ed None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:25.079242 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-365ca570-bf29-4f53-a8cd-729e0344a2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:25.138586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69b4d8-a007-4b52-8684-3dbc88a88e85 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:25.166708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 15/29] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:25.228123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:25.300635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69b4d8-a007-4b52-8684-3dbc88a88e85 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:25.345696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:25.346674 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:25.347151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:25.347650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:25.363731 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:25.365499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:34:25.384588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:25.385489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:34:25.386308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:34:25.386308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:34:25.386308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:25.388044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 15/30] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:34:25.393954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ca570-bf29-4f53-a8cd-729e0344a2ed None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:25.413676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:25.573270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ca570-bf29-4f53-a8cd-729e0344a2ed None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:25.595635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:25.601384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af155e-f724-4c22-8465-768a94a9eea5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:25.617543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:25.619213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:25.619318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:25.619555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:25.620202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:25.621023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:25.623157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:25.625353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:25.645985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 16/31] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:25.647807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:25.648462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f00f20-fd57-4407-aaa2-8d032e0be455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:25.649335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f00f20-fd57-4407-aaa2-8d032e0be455 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:25.649740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f00f20-fd57-4407-aaa2-8d032e0be455 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:25.650150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f00f20-fd57-4407-aaa2-8d032e0be455 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:25.650925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f00f20-fd57-4407-aaa2-8d032e0be455 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:25.654948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f00f20-fd57-4407-aaa2-8d032e0be455 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:25.763142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dd900-d3d2-4648-be17-5a746b6f03be None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:25.799909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ec564-f405-48af-be11-44eb00733c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:25.800850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ec564-f405-48af-be11-44eb00733c9e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:25.801330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ec564-f405-48af-be11-44eb00733c9e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:25.801749 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ec564-f405-48af-be11-44eb00733c9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:25.803975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 16/32] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 01:34:25 2022] POST /identity/v3/roles => generated 219 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:34:25.805179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f00f20-fd57-4407-aaa2-8d032e0be455 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:25.818694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ec564-f405-48af-be11-44eb00733c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-8d9ec564-f405-48af-be11-44eb00733c9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:25.827195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:25.828098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:25.828893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:25.831612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 17/33] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:34:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:25.832728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ec564-f405-48af-be11-44eb00733c9e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:25.904625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:25.945488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f00f20-fd57-4407-aaa2-8d032e0be455 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:25.959089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ec564-f405-48af-be11-44eb00733c9e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:26.002560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d58b4-9522-45c6-bd7a-d194d47a21e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:26.003764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d58b4-9522-45c6-bd7a-d194d47a21e7 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.005696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d58b4-9522-45c6-bd7a-d194d47a21e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.006192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d58b4-9522-45c6-bd7a-d194d47a21e7 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.007181 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d58b4-9522-45c6-bd7a-d194d47a21e7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:26.009954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d58b4-9522-45c6-bd7a-d194d47a21e7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:26.025177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a571c-c6cb-47fa-8d47-a8e713e16935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:26.026277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a571c-c6cb-47fa-8d47-a8e713e16935 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.027416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a571c-c6cb-47fa-8d47-a8e713e16935 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.027934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a571c-c6cb-47fa-8d47-a8e713e16935 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.028848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a571c-c6cb-47fa-8d47-a8e713e16935 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:26.030662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a571c-c6cb-47fa-8d47-a8e713e16935 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:26.040134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 17/34] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 01:34:25 2022] POST /identity/v3/roles => generated 225 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:34:26.041418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d58b4-9522-45c6-bd7a-d194d47a21e7 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:26.079929 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:26.176793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d58b4-9522-45c6-bd7a-d194d47a21e7 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:26.206750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 18/35] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:34:25 2022] POST /identity/v3/projects => generated 301 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:34:26.209394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877a0c6-e0de-43d3-ab7d-532ac6a54f53 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.209394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877a0c6-e0de-43d3-ab7d-532ac6a54f53 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.209394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877a0c6-e0de-43d3-ab7d-532ac6a54f53 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.210518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 19/36] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:26.211678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9351d44-ac94-4312-9880-89efcbdeb37f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.212031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9351d44-ac94-4312-9880-89efcbdeb37f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.212317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9351d44-ac94-4312-9880-89efcbdeb37f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.214664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 20/37] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:26.218577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfafd-2631-4bbb-adcf-0465bb565e5c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.219186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfafd-2631-4bbb-adcf-0465bb565e5c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.219472 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4cfafd-2631-4bbb-adcf-0465bb565e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.219660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609e695-bb6e-4bbd-bfb0-01b3e4217c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:26.220294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e695-bb6e-4bbd-bfb0-01b3e4217c98 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.220571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e695-bb6e-4bbd-bfb0-01b3e4217c98 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.220842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e695-bb6e-4bbd-bfb0-01b3e4217c98 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.221564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609e695-bb6e-4bbd-bfb0-01b3e4217c98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:26.221871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609e695-bb6e-4bbd-bfb0-01b3e4217c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:26.223639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609e695-bb6e-4bbd-bfb0-01b3e4217c98 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:26.238018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 18/38] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:26.239474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c819b-9a1e-4220-bb56-f12c0fb58853 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.239761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c819b-9a1e-4220-bb56-f12c0fb58853 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.240034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c819b-9a1e-4220-bb56-f12c0fb58853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.242243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6c4cfafd-2631-4bbb-adcf-0465bb565e5c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:26.244095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6c4cfafd-2631-4bbb-adcf-0465bb565e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:26.263787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5f0c819b-9a1e-4220-bb56-f12c0fb58853 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:26.265709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5f0c819b-9a1e-4220-bb56-f12c0fb58853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:26.349622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 21/39] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:26.349622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cfafd-2631-4bbb-adcf-0465bb565e5c None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:26.371356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 19/40] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:26.395938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:26.470881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4cfafd-2631-4bbb-adcf-0465bb565e5c None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:26.611830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2e524-436e-4bdb-ba01-ca605d232c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:26.612375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e524-436e-4bdb-ba01-ca605d232c08 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.612657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e524-436e-4bdb-ba01-ca605d232c08 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.612931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2e524-436e-4bdb-ba01-ca605d232c08 None admin] PATH_INFO: `/v3/users/2a2e93b317db4de6850826f625ccc3db` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.614082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2e524-436e-4bdb-ba01-ca605d232c08 None admin] RBAC: Authorizing `identity:get_user(user_id=2a2e93b317db4de6850826f625ccc3db)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:26.615634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2e524-436e-4bdb-ba01-ca605d232c08 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:26.617397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 22/41] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:26 2022] GET /identity/v3/users/2a2e93b317db4de6850826f625ccc3db => generated 247 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:26.622691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c819b-9a1e-4220-bb56-f12c0fb58853 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:26.624610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2e524-436e-4bdb-ba01-ca605d232c08 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:26.635284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:26.726895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2e524-436e-4bdb-ba01-ca605d232c08 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:26.727555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c819b-9a1e-4220-bb56-f12c0fb58853 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:26.729243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:26.760646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a7ccc-f064-4e85-84ec-64c4dbf80bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:26.761821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7ccc-f064-4e85-84ec-64c4dbf80bcc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.761920 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ae715-f2f6-4d53-9430-dfe82d18808b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:26.762106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7ccc-f064-4e85-84ec-64c4dbf80bcc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.762379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a7ccc-f064-4e85-84ec-64c4dbf80bcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.762505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ae715-f2f6-4d53-9430-dfe82d18808b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.762781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ae715-f2f6-4d53-9430-dfe82d18808b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.763049 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ae715-f2f6-4d53-9430-dfe82d18808b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.763500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a7ccc-f064-4e85-84ec-64c4dbf80bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:26.764430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ae715-f2f6-4d53-9430-dfe82d18808b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:26.764945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a7ccc-f064-4e85-84ec-64c4dbf80bcc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:26.765895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ae715-f2f6-4d53-9430-dfe82d18808b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:26.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 23/42] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:26.767296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a7ccc-f064-4e85-84ec-64c4dbf80bcc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:26.772222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 20/43] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:26.772909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ae715-f2f6-4d53-9430-dfe82d18808b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:26.851670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:26.865262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a7ccc-f064-4e85-84ec-64c4dbf80bcc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:26.874248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ae715-f2f6-4d53-9430-dfe82d18808b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:26.899511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467a67-7fcb-437b-b725-f69ac97c4942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:26.900123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467a67-7fcb-437b-b725-f69ac97c4942 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.900407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467a67-7fcb-437b-b725-f69ac97c4942 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.900684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43467a67-7fcb-437b-b725-f69ac97c4942 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.901386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43467a67-7fcb-437b-b725-f69ac97c4942 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:26.902805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43467a67-7fcb-437b-b725-f69ac97c4942 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:26.906644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22588914-af17-433b-9178-d2bf2fbf7112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:26.907257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22588914-af17-433b-9178-d2bf2fbf7112 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:26.907534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22588914-af17-433b-9178-d2bf2fbf7112 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:26.907808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22588914-af17-433b-9178-d2bf2fbf7112 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:26.908454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22588914-af17-433b-9178-d2bf2fbf7112 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:26.909909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22588914-af17-433b-9178-d2bf2fbf7112 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:26.919150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 24/44] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 01:34:26 2022] POST /identity/v3/roles => generated 223 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:34:26.920010 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467a67-7fcb-437b-b725-f69ac97c4942 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:26.947433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 21/45] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:34:26 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:34:26.975140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:27.063648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43467a67-7fcb-437b-b725-f69ac97c4942 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:27.104226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eae32e-7bae-4a99-89c8-d166f8d99d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:27.105033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eae32e-7bae-4a99-89c8-d166f8d99d15 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:27.105289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eae32e-7bae-4a99-89c8-d166f8d99d15 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:27.105566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eae32e-7bae-4a99-89c8-d166f8d99d15 None admin] PATH_INFO: `/v3/domains/default/users/2a2e93b317db4de6850826f625ccc3db/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:27.108019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eae32e-7bae-4a99-89c8-d166f8d99d15 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2a2e93b317db4de6850826f625ccc3db, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:27.109168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eae32e-7bae-4a99-89c8-d166f8d99d15 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:27.125079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 25/46] 10.222.0.15 () {58 vars in 1361 bytes} [Fri Sep 9 01:34:26 2022] PUT /identity/v3/domains/default/users/2a2e93b317db4de6850826f625ccc3db/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:34:29.133391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46b22-a7d9-44dd-bd5f-d9c935624e22 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:29.133954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46b22-a7d9-44dd-bd5f-d9c935624e22 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:29.133954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec46b22-a7d9-44dd-bd5f-d9c935624e22 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:29.135318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 22/47] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:29.141594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acdc6a-b26c-4748-ae1f-8fec4b4523b1 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:29.142109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acdc6a-b26c-4748-ae1f-8fec4b4523b1 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:29.142227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acdc6a-b26c-4748-ae1f-8fec4b4523b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:29.160490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-30acdc6a-b26c-4748-ae1f-8fec4b4523b1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:29.161651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-30acdc6a-b26c-4748-ae1f-8fec4b4523b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:29.226954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 26/48] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:29.238337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b833f5-c0ab-4aeb-8d20-32e330086442 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:29.238622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b833f5-c0ab-4aeb-8d20-32e330086442 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:29.238897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b833f5-c0ab-4aeb-8d20-32e330086442 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:29.240219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 23/49] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:29.245994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5104b36-ce79-47bd-bbaa-1ed1a1e44659 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:29.246291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5104b36-ce79-47bd-bbaa-1ed1a1e44659 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:29.246574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5104b36-ce79-47bd-bbaa-1ed1a1e44659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:29.264429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a5104b36-ce79-47bd-bbaa-1ed1a1e44659 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:29.265670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a5104b36-ce79-47bd-bbaa-1ed1a1e44659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:29.281703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:29.332439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 27/50] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:29.493987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b833f5-c0ab-4aeb-8d20-32e330086442 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:29.591077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b833f5-c0ab-4aeb-8d20-32e330086442 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:29.603554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5104b36-ce79-47bd-bbaa-1ed1a1e44659 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:29.620175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:29.626691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589f09c-55d9-4033-aef6-6dc44e049863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:29.627610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589f09c-55d9-4033-aef6-6dc44e049863 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:29.628034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589f09c-55d9-4033-aef6-6dc44e049863 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:29.628519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589f09c-55d9-4033-aef6-6dc44e049863 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:29.629654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589f09c-55d9-4033-aef6-6dc44e049863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:29.631329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589f09c-55d9-4033-aef6-6dc44e049863 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:29.632885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 24/51] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:29.638244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589f09c-55d9-4033-aef6-6dc44e049863 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:29.702580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5104b36-ce79-47bd-bbaa-1ed1a1e44659 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:29.736427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589f09c-55d9-4033-aef6-6dc44e049863 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:29.736841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11d44c-6b4c-45f9-a4c2-26d755431d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:29.737500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11d44c-6b4c-45f9-a4c2-26d755431d9a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:29.737790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11d44c-6b4c-45f9-a4c2-26d755431d9a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:29.738064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11d44c-6b4c-45f9-a4c2-26d755431d9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:29.748015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11d44c-6b4c-45f9-a4c2-26d755431d9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-0d11d44c-6b4c-45f9-a4c2-26d755431d9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:29.752805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:29.753660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:29.754708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 28/52] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:34:29.759885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11d44c-6b4c-45f9-a4c2-26d755431d9a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:29.774088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6d2e0-1422-44a3-8a37-1252a17e49d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:29.774707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d2e0-1422-44a3-8a37-1252a17e49d0 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:29.775005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d2e0-1422-44a3-8a37-1252a17e49d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:29.775262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6d2e0-1422-44a3-8a37-1252a17e49d0 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:29.775910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6d2e0-1422-44a3-8a37-1252a17e49d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:29.777957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6d2e0-1422-44a3-8a37-1252a17e49d0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:29.779634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:29.804473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 25/53] 10.222.0.15 () {60 vars in 1140 bytes} [Fri Sep 9 01:34:29 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:34:29.871512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11d44c-6b4c-45f9-a4c2-26d755431d9a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:29.908653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:29.909302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:29.909587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:29.909861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:29.910552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:29.910859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:29.912263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:29.924628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 29/54] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:29.930680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6d2e0-1422-44a3-8a37-1252a17e49d0 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:29.982043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:30.035577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6d2e0-1422-44a3-8a37-1252a17e49d0 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:30.068311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761b20-457c-4a1e-9417-3bb87c06ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:30.068941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761b20-457c-4a1e-9417-3bb87c06ab77 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:30.069250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761b20-457c-4a1e-9417-3bb87c06ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:30.069528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e761b20-457c-4a1e-9417-3bb87c06ab77 None admin] PATH_INFO: `/v3/users/2a2e93b317db4de6850826f625ccc3db` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:30.070867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e761b20-457c-4a1e-9417-3bb87c06ab77 None admin] RBAC: Authorizing `identity:get_user(user_id=2a2e93b317db4de6850826f625ccc3db)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:30.072323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e761b20-457c-4a1e-9417-3bb87c06ab77 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:30.073984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 26/55] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:29 2022] GET /identity/v3/users/2a2e93b317db4de6850826f625ccc3db => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:30.079448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:30.158215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:30.183348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa8ba1-9ed5-451e-85ce-96eee75253a9 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:30.216919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b61eb-fbbd-4478-b60e-31fc6f832717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:30.217576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b61eb-fbbd-4478-b60e-31fc6f832717 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:30.217860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b61eb-fbbd-4478-b60e-31fc6f832717 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:30.218141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b61eb-fbbd-4478-b60e-31fc6f832717 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:30.219255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b61eb-fbbd-4478-b60e-31fc6f832717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:30.220723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b61eb-fbbd-4478-b60e-31fc6f832717 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:30.222346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 30/56] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:30.228130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761b20-457c-4a1e-9417-3bb87c06ab77 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:30.326694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e761b20-457c-4a1e-9417-3bb87c06ab77 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:30.342828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:30.359400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c110f-8190-47b9-8451-0424cc474b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:30.360008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c110f-8190-47b9-8451-0424cc474b01 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:30.360289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c110f-8190-47b9-8451-0424cc474b01 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:30.360566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c110f-8190-47b9-8451-0424cc474b01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:30.361316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c110f-8190-47b9-8451-0424cc474b01 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.domain.id=default, user.id=2a2e93b317db4de6850826f625ccc3db) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:30.361630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c110f-8190-47b9-8451-0424cc474b01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:30.363073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c110f-8190-47b9-8451-0424cc474b01 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:30.372434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 27/57] 10.222.0.15 () {56 vars in 1322 bytes} [Fri Sep 9 01:34:30 2022] GET /identity/v3/role_assignments?user.id=2a2e93b317db4de6850826f625ccc3db&scope.domain.id=default&role.id=b131b61299154544a627315649090a49 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:31.961713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422d047-cccb-4e1e-bd26-82eca840e010 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:31.962156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422d047-cccb-4e1e-bd26-82eca840e010 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:31.962278 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422d047-cccb-4e1e-bd26-82eca840e010 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:31.963639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 31/58] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:31.969296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96e2d-600b-4151-9392-fb91a7b03d03 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:31.969655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96e2d-600b-4151-9392-fb91a7b03d03 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:31.969846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d96e2d-600b-4151-9392-fb91a7b03d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:31.987711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-19d96e2d-600b-4151-9392-fb91a7b03d03 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:31.988900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-19d96e2d-600b-4151-9392-fb91a7b03d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:32.053579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 28/59] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:32.321290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422d047-cccb-4e1e-bd26-82eca840e010 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:32.418586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422d047-cccb-4e1e-bd26-82eca840e010 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:32.450564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d9a1-2eb0-44da-8356-023bbc9222de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:32.451172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d9a1-2eb0-44da-8356-023bbc9222de None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:32.451448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d9a1-2eb0-44da-8356-023bbc9222de None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:32.451716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d9a1-2eb0-44da-8356-023bbc9222de None admin] PATH_INFO: `/v3/domains/default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:32.452809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d9a1-2eb0-44da-8356-023bbc9222de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:32.454276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d9a1-2eb0-44da-8356-023bbc9222de None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:32.455796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 32/60] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:32.461139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d96e2d-600b-4151-9392-fb91a7b03d03 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:32.662790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d96e2d-600b-4151-9392-fb91a7b03d03 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:32.675414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:32.695487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe0ca9-dd11-4cc6-b293-6889b78a414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:32.696079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe0ca9-dd11-4cc6-b293-6889b78a414d None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:32.696355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe0ca9-dd11-4cc6-b293-6889b78a414d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:32.696626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe0ca9-dd11-4cc6-b293-6889b78a414d None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:32.697287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe0ca9-dd11-4cc6-b293-6889b78a414d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:32.699211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe0ca9-dd11-4cc6-b293-6889b78a414d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:32.725221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 29/61] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:34:32 2022] POST /identity/v3/users => generated 275 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:34:36.087479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df59d-d77b-4151-a963-474331a3f728 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.088507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df59d-d77b-4151-a963-474331a3f728 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.089531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df59d-d77b-4151-a963-474331a3f728 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.091082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 33/62] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:36.096756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2c896-eab7-4f8a-abab-0d9dc928313b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.097628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2c896-eab7-4f8a-abab-0d9dc928313b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.098099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2c896-eab7-4f8a-abab-0d9dc928313b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.121795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4fd2c896-eab7-4f8a-abab-0d9dc928313b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:36.123373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4fd2c896-eab7-4f8a-abab-0d9dc928313b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:36.128947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4d21c-900e-430a-9ffe-8df94454d72b None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.129279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4d21c-900e-430a-9ffe-8df94454d72b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.129560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4d21c-900e-430a-9ffe-8df94454d72b None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.131175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 34/63] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:36.138198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95fe3-d1e1-4c51-b0fa-d58f45ef21b9 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.138709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95fe3-d1e1-4c51-b0fa-d58f45ef21b9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.139193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95fe3-d1e1-4c51-b0fa-d58f45ef21b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.165461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9bf95fe3-d1e1-4c51-b0fa-d58f45ef21b9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:36.169809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9bf95fe3-d1e1-4c51-b0fa-d58f45ef21b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:36.175784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:36.261757 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 30/64] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:36.308057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 35/65] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:36.382991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a44c-666d-4410-9b3b-ff6213985748 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.383289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a44c-666d-4410-9b3b-ff6213985748 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.383537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a44c-666d-4410-9b3b-ff6213985748 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.385168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 31/66] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:36.390169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50fd1-af85-49bd-b856-8b86d7995760 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.390461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50fd1-af85-49bd-b856-8b86d7995760 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.390745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50fd1-af85-49bd-b856-8b86d7995760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.418127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a2c50fd1-af85-49bd-b856-8b86d7995760 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:36.419240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a2c50fd1-af85-49bd-b856-8b86d7995760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:36.507152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 36/67] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:36.536011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192a44c-666d-4410-9b3b-ff6213985748 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:36.636683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c50fd1-af85-49bd-b856-8b86d7995760 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:36.700371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:36.705543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192a44c-666d-4410-9b3b-ff6213985748 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:36.726093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:36.744282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e35b33-bfb9-4d6a-9b59-ac8f706eb0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:36.744910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e35b33-bfb9-4d6a-9b59-ac8f706eb0da None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.745828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e35b33-bfb9-4d6a-9b59-ac8f706eb0da None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.746100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e35b33-bfb9-4d6a-9b59-ac8f706eb0da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.757782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e35b33-bfb9-4d6a-9b59-ac8f706eb0da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:36.759528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c50fd1-af85-49bd-b856-8b86d7995760 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-42e35b33-bfb9-4d6a-9b59-ac8f706eb0da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:36.764389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:36.765269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:36.766017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 32/68] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:36.766761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb96345-6742-4133-9245-106532581838 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.766975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb96345-6742-4133-9245-106532581838 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.767215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb96345-6742-4133-9245-106532581838 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.768453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 33/69] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:36.769471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dac62-cc9a-4595-8516-d78e1c5f8eda None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.769749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dac62-cc9a-4595-8516-d78e1c5f8eda None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.770016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dac62-cc9a-4595-8516-d78e1c5f8eda None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.771245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 34/70] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:36.776101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dac62-cc9a-4595-8516-d78e1c5f8eda None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:36.799338 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fe876-e73b-4d8f-b0dd-9cb532b13b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:36.800172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fe876-e73b-4d8f-b0dd-9cb532b13b9a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.800634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fe876-e73b-4d8f-b0dd-9cb532b13b9a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.801179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fe876-e73b-4d8f-b0dd-9cb532b13b9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.812179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fe876-e73b-4d8f-b0dd-9cb532b13b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-ec3fe876-e73b-4d8f-b0dd-9cb532b13b9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:36.817219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:36.818074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:36.818820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 37/71] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:36.820059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089876-044b-4a7a-a909-f32dae0e4ece None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.820292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089876-044b-4a7a-a909-f32dae0e4ece None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.820664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80089876-044b-4a7a-a909-f32dae0e4ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.837816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-80089876-044b-4a7a-a909-f32dae0e4ece None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:36.839533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-80089876-044b-4a7a-a909-f32dae0e4ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:36.888333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dac62-cc9a-4595-8516-d78e1c5f8eda None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:36.906621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 38/72] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:36.908035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faf9b5-fd56-4f35-b875-2e706103d3ae None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.908326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faf9b5-fd56-4f35-b875-2e706103d3ae None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.908603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faf9b5-fd56-4f35-b875-2e706103d3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.927366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-47faf9b5-fd56-4f35-b875-2e706103d3ae None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:36.928549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-47faf9b5-fd56-4f35-b875-2e706103d3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:36.929602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:36.930289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:36.930577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:36.930842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:36.932217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:36.932217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:36.933737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:36.942748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 35/73] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:36.943393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:36.967868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:36.996999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:37.009917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 39/74] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:37.021911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faf9b5-fd56-4f35-b875-2e706103d3ae None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.051268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30261e-4964-4cf1-89dc-d639fe03cddf None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.087542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.088252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.088543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.088875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.089846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:37.089954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:37.091322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:37.100023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 36/75] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:37.100815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.136172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faf9b5-fd56-4f35-b875-2e706103d3ae None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.174199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bd271-924f-4dec-90ba-879e7b1c55f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.175165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bd271-924f-4dec-90ba-879e7b1c55f8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.175646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bd271-924f-4dec-90ba-879e7b1c55f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.176119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bd271-924f-4dec-90ba-879e7b1c55f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.186539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:37.187719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35bd271-924f-4dec-90ba-879e7b1c55f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-b35bd271-924f-4dec-90ba-879e7b1c55f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:37.192719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:37.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:37.194488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 40/76] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:37.195095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bd271-924f-4dec-90ba-879e7b1c55f8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.211654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80660efd-2891-48e2-a3b9-7bb771d309e6 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.249571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:37.254168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af05d31-75aa-43a9-89f2-5d2744f03816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.255087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05d31-75aa-43a9-89f2-5d2744f03816 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.255381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05d31-75aa-43a9-89f2-5d2744f03816 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.255661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af05d31-75aa-43a9-89f2-5d2744f03816 None admin] PATH_INFO: `/v3/users/2a2e93b317db4de6850826f625ccc3db` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.256822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af05d31-75aa-43a9-89f2-5d2744f03816 None admin] RBAC: Authorizing `identity:get_user(user_id=2a2e93b317db4de6850826f625ccc3db)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:37.258297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af05d31-75aa-43a9-89f2-5d2744f03816 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:37.259888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 37/77] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/users/2a2e93b317db4de6850826f625ccc3db => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:37.261184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af05d31-75aa-43a9-89f2-5d2744f03816 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.305710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bd271-924f-4dec-90ba-879e7b1c55f8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.339731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af0ce6-61ff-4263-82da-0d05fd1af37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.340343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0ce6-61ff-4263-82da-0d05fd1af37c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.340621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0ce6-61ff-4263-82da-0d05fd1af37c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.340891 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0ce6-61ff-4263-82da-0d05fd1af37c None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.353465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af0ce6-61ff-4263-82da-0d05fd1af37c None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:37.354981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af0ce6-61ff-4263-82da-0d05fd1af37c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:37.356556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 41/78] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:37.361793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af0ce6-61ff-4263-82da-0d05fd1af37c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.362556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af05d31-75aa-43a9-89f2-5d2744f03816 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.397813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082aadd-1a49-4917-bbae-7dcc4b592eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.398483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082aadd-1a49-4917-bbae-7dcc4b592eef None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.398762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082aadd-1a49-4917-bbae-7dcc4b592eef None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.399032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082aadd-1a49-4917-bbae-7dcc4b592eef None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.400519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082aadd-1a49-4917-bbae-7dcc4b592eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:37.402458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082aadd-1a49-4917-bbae-7dcc4b592eef None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:37.404473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 38/79] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:37.405101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082aadd-1a49-4917-bbae-7dcc4b592eef None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.423490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:37.462437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af0ce6-61ff-4263-82da-0d05fd1af37c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.471377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:37.498235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.498838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.499114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.499393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.500095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:37.500345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:37.501733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:37.502775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082aadd-1a49-4917-bbae-7dcc4b592eef None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.509727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 42/80] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:37.509727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.548832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5017-d1f5-4003-a513-96eab789680b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.549867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5017-d1f5-4003-a513-96eab789680b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.550156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5017-d1f5-4003-a513-96eab789680b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.550434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5017-d1f5-4003-a513-96eab789680b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.612038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45807a5-0c48-4ee6-9649-78f5572b60b4 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.623384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f5017-d1f5-4003-a513-96eab789680b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:37.624866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f5017-d1f5-4003-a513-96eab789680b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:37.642535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:37.646616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38872118-d247-4720-8058-dde8f99a2204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.647253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38872118-d247-4720-8058-dde8f99a2204 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.647536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38872118-d247-4720-8058-dde8f99a2204 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.647810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38872118-d247-4720-8058-dde8f99a2204 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.657859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38872118-d247-4720-8058-dde8f99a2204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-38872118-d247-4720-8058-dde8f99a2204 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:37.663129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:37.664015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:37.664760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 43/81] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:37.665842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38872118-d247-4720-8058-dde8f99a2204 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.695528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:37.726977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 39/82] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:37.727800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5017-d1f5-4003-a513-96eab789680b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.768201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38872118-d247-4720-8058-dde8f99a2204 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.800841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624abb8e-be64-487d-9429-904bf83db407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.801484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624abb8e-be64-487d-9429-904bf83db407 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.801760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624abb8e-be64-487d-9429-904bf83db407 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.802031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624abb8e-be64-487d-9429-904bf83db407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.823304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5017-d1f5-4003-a513-96eab789680b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:37.841092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:37.865520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753bf14-8ec6-48ee-85e9-5d9ddddffc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:37.866299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753bf14-8ec6-48ee-85e9-5d9ddddffc4c None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:37.866584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753bf14-8ec6-48ee-85e9-5d9ddddffc4c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:37.866862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753bf14-8ec6-48ee-85e9-5d9ddddffc4c None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:37.867522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753bf14-8ec6-48ee-85e9-5d9ddddffc4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:37.868981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753bf14-8ec6-48ee-85e9-5d9ddddffc4c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:37.875063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624abb8e-be64-487d-9429-904bf83db407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:37.876765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624abb8e-be64-487d-9429-904bf83db407 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:37.892240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 40/83] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:34:37 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:34:37.893110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753bf14-8ec6-48ee-85e9-5d9ddddffc4c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:37.909584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:38.001614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 44/84] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:38.001614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624abb8e-be64-487d-9429-904bf83db407 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.028432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753bf14-8ec6-48ee-85e9-5d9ddddffc4c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.068822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b5958-cb1c-41ca-a6b6-4400fe910b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.069505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b5958-cb1c-41ca-a6b6-4400fe910b16 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.070581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b5958-cb1c-41ca-a6b6-4400fe910b16 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.070581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b5958-cb1c-41ca-a6b6-4400fe910b16 None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.072021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b5958-cb1c-41ca-a6b6-4400fe910b16 None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.075132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b5958-cb1c-41ca-a6b6-4400fe910b16 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:38.076585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 41/85] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:38.076952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b5958-cb1c-41ca-a6b6-4400fe910b16 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.116758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624abb8e-be64-487d-9429-904bf83db407 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.123480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:38.152500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:38.164514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.165244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.165529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.165806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.166478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:38.166783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.168222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:38.179251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 45/86] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:38.180926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.194175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b5958-cb1c-41ca-a6b6-4400fe910b16 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.242640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb7d88-6cc2-4e03-a806-97d2e3fef08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.243276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7d88-6cc2-4e03-a806-97d2e3fef08c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.243555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7d88-6cc2-4e03-a806-97d2e3fef08c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.243829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7d88-6cc2-4e03-a806-97d2e3fef08c None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.245026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb7d88-6cc2-4e03-a806-97d2e3fef08c None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.246518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb7d88-6cc2-4e03-a806-97d2e3fef08c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:38.248153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 42/87] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:38 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:38.248937 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb7d88-6cc2-4e03-a806-97d2e3fef08c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.312625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11526e-4416-47dc-82d0-671dd46adf96 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.349551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66656c7a-1135-43bb-8aed-95924a49742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.350331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66656c7a-1135-43bb-8aed-95924a49742c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.350658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66656c7a-1135-43bb-8aed-95924a49742c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.351004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66656c7a-1135-43bb-8aed-95924a49742c None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.352236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66656c7a-1135-43bb-8aed-95924a49742c None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.353821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66656c7a-1135-43bb-8aed-95924a49742c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:38.356444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 46/88] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:38 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:38.357706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66656c7a-1135-43bb-8aed-95924a49742c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.372883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb7d88-6cc2-4e03-a806-97d2e3fef08c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.397435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:38.411236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3375a3-577b-4894-af7f-d55ae6579484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.412346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3375a3-577b-4894-af7f-d55ae6579484 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.412728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3375a3-577b-4894-af7f-d55ae6579484 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.413194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3375a3-577b-4894-af7f-d55ae6579484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.431090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:38.484008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66656c7a-1135-43bb-8aed-95924a49742c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.493777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3375a3-577b-4894-af7f-d55ae6579484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.496110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3375a3-577b-4894-af7f-d55ae6579484 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:38.521474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe7f36-be71-498b-8a5e-8288580fc8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.522298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7f36-be71-498b-8a5e-8288580fc8ee None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.522527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7f36-be71-498b-8a5e-8288580fc8ee None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.523155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe7f36-be71-498b-8a5e-8288580fc8ee None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.524332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe7f36-be71-498b-8a5e-8288580fc8ee None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.525878 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe7f36-be71-498b-8a5e-8288580fc8ee None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:38.532603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 47/89] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:38 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:38.532603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe7f36-be71-498b-8a5e-8288580fc8ee None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.629096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 43/90] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:38.629096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3375a3-577b-4894-af7f-d55ae6579484 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.640597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:38.648094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe7f36-be71-498b-8a5e-8288580fc8ee None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.682772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:38.693440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.694194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.694601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.694887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.695622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=b6271cacfed747da992ff32e5f4813f9, user.id=2a2e93b317db4de6850826f625ccc3db) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:38.695934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.698756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:38.706611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 48/91] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:38 2022] GET /identity/v3/role_assignments?user.id=2a2e93b317db4de6850826f625ccc3db&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=b131b61299154544a627315649090a49 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:38.707413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.764268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3375a3-577b-4894-af7f-d55ae6579484 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.841807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1ba18-bcb9-4891-abf2-87d29ec2c865 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:38.862540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.863588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.864064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.864525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.865507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] RBAC: Adding query filter params (role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=b6271cacfed747da992ff32e5f4813f9, user.id=9190860e1df04a81b08ae447688ed74d) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:38.866039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.867708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:38.882957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4a0c5-96f6-4382-a372-7ba40114b20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:38.883499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4a0c5-96f6-4382-a372-7ba40114b20d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:38.883782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4a0c5-96f6-4382-a372-7ba40114b20d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:38.884059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4a0c5-96f6-4382-a372-7ba40114b20d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:38.885230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 44/92] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:38 2022] GET /identity/v3/role_assignments?user.id=9190860e1df04a81b08ae447688ed74d&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=18bebd2151824cdfa386be482ff8114b => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:38.887070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:38.970137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:38.984513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4a0c5-96f6-4382-a372-7ba40114b20d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:38.987200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4a0c5-96f6-4382-a372-7ba40114b20d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:39.129550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 49/93] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:39.131548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:39.144502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4a0c5-96f6-4382-a372-7ba40114b20d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:39.203113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5037e-8a85-4736-a511-413185aa0929 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:39.265319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:39.274344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4a0c5-96f6-4382-a372-7ba40114b20d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:39.293782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09792ec-256f-41b0-a03d-7058cfd7956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:39.294763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09792ec-256f-41b0-a03d-7058cfd7956a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:39.295244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09792ec-256f-41b0-a03d-7058cfd7956a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:39.295714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09792ec-256f-41b0-a03d-7058cfd7956a None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:39.297813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09792ec-256f-41b0-a03d-7058cfd7956a None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:39.300231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09792ec-256f-41b0-a03d-7058cfd7956a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:39.309198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 45/94] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:38 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:39.314063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09792ec-256f-41b0-a03d-7058cfd7956a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:39.329931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11b63f-e1fd-411f-bda7-6aba74032ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:39.330596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11b63f-e1fd-411f-bda7-6aba74032ebd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:39.330881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11b63f-e1fd-411f-bda7-6aba74032ebd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:39.331156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11b63f-e1fd-411f-bda7-6aba74032ebd None admin] PATH_INFO: `/v3/projects/ada241b36a134695b8f5258a0c7af039` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:39.332525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11b63f-e1fd-411f-bda7-6aba74032ebd None admin] RBAC: Authorizing `identity:get_project(project_id=ada241b36a134695b8f5258a0c7af039)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:39.334380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11b63f-e1fd-411f-bda7-6aba74032ebd None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:39.336368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 50/95] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:39 2022] GET /identity/v3/projects/ada241b36a134695b8f5258a0c7af039 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:39.353233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11b63f-e1fd-411f-bda7-6aba74032ebd None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:39.588046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11b63f-e1fd-411f-bda7-6aba74032ebd None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:39.598301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09792ec-256f-41b0-a03d-7058cfd7956a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:39.663289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303282-0ed0-43b3-878e-d52d9db50887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:39.663911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303282-0ed0-43b3-878e-d52d9db50887 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:39.664192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303282-0ed0-43b3-878e-d52d9db50887 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:39.664467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303282-0ed0-43b3-878e-d52d9db50887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:39.665227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df303282-0ed0-43b3-878e-d52d9db50887 None admin] RBAC: Adding query filter params (role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=ada241b36a134695b8f5258a0c7af039, user.id=9190860e1df04a81b08ae447688ed74d) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:39.665659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df303282-0ed0-43b3-878e-d52d9db50887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:39.667768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df303282-0ed0-43b3-878e-d52d9db50887 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:39.683365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 51/96] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:39 2022] GET /identity/v3/role_assignments?user.id=9190860e1df04a81b08ae447688ed74d&scope.project.id=ada241b36a134695b8f5258a0c7af039&role.id=18bebd2151824cdfa386be482ff8114b => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:39.693013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d62a4-69e9-4991-8fd3-9573e2b44c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:39.693679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d62a4-69e9-4991-8fd3-9573e2b44c4a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:39.693949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d62a4-69e9-4991-8fd3-9573e2b44c4a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:39.697086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d62a4-69e9-4991-8fd3-9573e2b44c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:39.697086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d62a4-69e9-4991-8fd3-9573e2b44c4a None admin] RBAC: Adding query filter params (role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=b6271cacfed747da992ff32e5f4813f9, user.id=9190860e1df04a81b08ae447688ed74d) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:39.697086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d62a4-69e9-4991-8fd3-9573e2b44c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:39.697086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d62a4-69e9-4991-8fd3-9573e2b44c4a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:39.710508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 46/97] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:39 2022] GET /identity/v3/role_assignments?user.id=9190860e1df04a81b08ae447688ed74d&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=eb7bedf01e0744eeb26681978db4121c => generated 259 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:41.129776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4f6c2-17d6-4d6a-9a79-481789fbcd8f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:41.131024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4f6c2-17d6-4d6a-9a79-481789fbcd8f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:41.131524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4f6c2-17d6-4d6a-9a79-481789fbcd8f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:41.133340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 52/98] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:41.139020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8ad08-f4e0-4ccc-baf8-497d64696d2b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:41.139814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8ad08-f4e0-4ccc-baf8-497d64696d2b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:41.140282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8ad08-f4e0-4ccc-baf8-497d64696d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:41.153797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:41.162582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6aa8ad08-f4e0-4ccc-baf8-497d64696d2b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:41.163880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6aa8ad08-f4e0-4ccc-baf8-497d64696d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:41.175419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60111337-3ed3-435a-8c1d-0890f53a2f59 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:41.175697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60111337-3ed3-435a-8c1d-0890f53a2f59 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:41.175972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60111337-3ed3-435a-8c1d-0890f53a2f59 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:41.177341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 53/99] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:41.182557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe4a1-ec3b-4d2e-bf56-d115704b4d5b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:41.182840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe4a1-ec3b-4d2e-bf56-d115704b4d5b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:41.183112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe4a1-ec3b-4d2e-bf56-d115704b4d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:41.220537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8c7fe4a1-ec3b-4d2e-bf56-d115704b4d5b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:41.221757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8c7fe4a1-ec3b-4d2e-bf56-d115704b4d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:41.253553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:41.303565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 47/100] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:41.335861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 54/101] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:41.728368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8ad08-f4e0-4ccc-baf8-497d64696d2b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:41.830994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8ad08-f4e0-4ccc-baf8-497d64696d2b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:41.872472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129c001-feab-4c10-8850-aab75ec49d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:41.873145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c001-feab-4c10-8850-aab75ec49d19 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:41.873429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c001-feab-4c10-8850-aab75ec49d19 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:41.873703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129c001-feab-4c10-8850-aab75ec49d19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:41.874882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129c001-feab-4c10-8850-aab75ec49d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:41.876318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129c001-feab-4c10-8850-aab75ec49d19 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:41.878301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 48/102] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:41.878646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe4a1-ec3b-4d2e-bf56-d115704b4d5b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:41.897025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129c001-feab-4c10-8850-aab75ec49d19 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:41.901740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:42.017489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129c001-feab-4c10-8850-aab75ec49d19 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:42.056869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f02212-daf5-45c6-ad10-efbfb98f65f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:42.058668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f02212-daf5-45c6-ad10-efbfb98f65f8 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.059149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f02212-daf5-45c6-ad10-efbfb98f65f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.059642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f02212-daf5-45c6-ad10-efbfb98f65f8 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.060493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f02212-daf5-45c6-ad10-efbfb98f65f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:42.062856 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f02212-daf5-45c6-ad10-efbfb98f65f8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:42.073600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe4a1-ec3b-4d2e-bf56-d115704b4d5b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:42.096699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 49/103] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:34:41 2022] POST /identity/v3/projects => generated 302 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:34:42.098368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdbad3-65c6-4ff3-a7c2-405b2ded91cf None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.099373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdbad3-65c6-4ff3-a7c2-405b2ded91cf None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.099903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdbad3-65c6-4ff3-a7c2-405b2ded91cf None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.101808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 50/104] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:42.107498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584337-d717-439b-8768-edc77bef42fd None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.108518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584337-d717-439b-8768-edc77bef42fd None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.109533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77584337-d717-439b-8768-edc77bef42fd None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.111504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 51/105] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:42.112897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0c517-5f8f-4446-92ad-df930867a9ec None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.113420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0c517-5f8f-4446-92ad-df930867a9ec None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.113893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0c517-5f8f-4446-92ad-df930867a9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.117688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c2a15-2e29-43ad-bc38-3fc7854338cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:42.118701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2a15-2e29-43ad-bc38-3fc7854338cd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.119180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2a15-2e29-43ad-bc38-3fc7854338cd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.119787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2a15-2e29-43ad-bc38-3fc7854338cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.134473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c2a15-2e29-43ad-bc38-3fc7854338cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:42.138448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-74f0c517-5f8f-4446-92ad-df930867a9ec None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:42.139676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-74f0c517-5f8f-4446-92ad-df930867a9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-c72c2a15-2e29-43ad-bc38-3fc7854338cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:42.141630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:42.142506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:34:42.143246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:42.146857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 55/106] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:42.148469 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45dc5-cc6d-4ea8-8ed2-70e38e6267b0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.148974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45dc5-cc6d-4ea8-8ed2-70e38e6267b0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.149482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45dc5-cc6d-4ea8-8ed2-70e38e6267b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.187239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-43e45dc5-cc6d-4ea8-8ed2-70e38e6267b0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:42.188500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-43e45dc5-cc6d-4ea8-8ed2-70e38e6267b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:42.215973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:42.246907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:42.294797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 52/107] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:42.295393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0c517-5f8f-4446-92ad-df930867a9ec None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:42.323448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 56/108] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:42.429114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0c517-5f8f-4446-92ad-df930867a9ec None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:42.472221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:42.473271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.473753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.474224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.475082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:42.475662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:42.477289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:42.486550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 53/109] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:42.496141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e45dc5-cc6d-4ea8-8ed2-70e38e6267b0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:42.629541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:42.645126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e45dc5-cc6d-4ea8-8ed2-70e38e6267b0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:42.701459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:42.713273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4830-dd36-4e8a-aa89-3ac90b6a7162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:42.714635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4830-dd36-4e8a-aa89-3ac90b6a7162 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.714930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4830-dd36-4e8a-aa89-3ac90b6a7162 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.715199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c4830-dd36-4e8a-aa89-3ac90b6a7162 None admin] PATH_INFO: `/v3/users/ecbecebf1fc5466393a2a83eb2488e94` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.761188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c4830-dd36-4e8a-aa89-3ac90b6a7162 None admin] RBAC: Authorizing `identity:get_user(user_id=ecbecebf1fc5466393a2a83eb2488e94)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:42.762849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69c4830-dd36-4e8a-aa89-3ac90b6a7162 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:42.763928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acea47-a5b2-4410-800e-ea112009db99 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:42.764610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 57/110] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:42 2022] GET /identity/v3/users/ecbecebf1fc5466393a2a83eb2488e94 => generated 289 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:42.765412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4830-dd36-4e8a-aa89-3ac90b6a7162 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:42.772387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:42.803413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbdef8-623d-4fce-b02a-10831d0fb4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:42.804161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbdef8-623d-4fce-b02a-10831d0fb4a7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.804503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbdef8-623d-4fce-b02a-10831d0fb4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.804786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbdef8-623d-4fce-b02a-10831d0fb4a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:42.816127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbdef8-623d-4fce-b02a-10831d0fb4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-d9fbdef8-623d-4fce-b02a-10831d0fb4a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:42.821816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:42.822651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:42.823444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:42.823995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 54/111] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:34:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:34:42.824774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbdef8-623d-4fce-b02a-10831d0fb4a7 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:42.949079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbdef8-623d-4fce-b02a-10831d0fb4a7 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:42.962634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:42.969467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c4830-dd36-4e8a-aa89-3ac90b6a7162 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:42.995902 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0a238-7450-4026-a0aa-38da865e0988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:42.996709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0a238-7450-4026-a0aa-38da865e0988 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:42.997116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0a238-7450-4026-a0aa-38da865e0988 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:42.997489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0a238-7450-4026-a0aa-38da865e0988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.016054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112889e-b2fe-4f35-8b54-194b069880a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:43.017191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112889e-b2fe-4f35-8b54-194b069880a8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.017477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112889e-b2fe-4f35-8b54-194b069880a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.017898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112889e-b2fe-4f35-8b54-194b069880a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.032307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112889e-b2fe-4f35-8b54-194b069880a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-e112889e-b2fe-4f35-8b54-194b069880a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:43.039296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:43.040254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 58/112] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:43.041270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112889e-b2fe-4f35-8b54-194b069880a8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.079200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0a238-7450-4026-a0aa-38da865e0988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:43.080710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0a238-7450-4026-a0aa-38da865e0988 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:43.118233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:43.161481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112889e-b2fe-4f35-8b54-194b069880a8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:43.190010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 55/113] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:43.191291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474c63-83d3-439f-bb36-5f00900d2752 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.191633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474c63-83d3-439f-bb36-5f00900d2752 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.191916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474c63-83d3-439f-bb36-5f00900d2752 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.193190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 56/114] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:43.193886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c474c63-83d3-439f-bb36-5f00900d2752 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.202271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf2235-c940-40a5-98f6-8db3425f4bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:43.203161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf2235-c940-40a5-98f6-8db3425f4bda None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.203634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf2235-c940-40a5-98f6-8db3425f4bda None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.204118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf2235-c940-40a5-98f6-8db3425f4bda None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.204967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf2235-c940-40a5-98f6-8db3425f4bda None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:43.205421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf2235-c940-40a5-98f6-8db3425f4bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:43.207634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf2235-c940-40a5-98f6-8db3425f4bda None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:43.208801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:43.218927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 59/115] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:43.220124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3227-9212-460a-a83b-5e398552be59 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.220415 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3227-9212-460a-a83b-5e398552be59 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.220694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f3227-9212-460a-a83b-5e398552be59 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.225310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 60/116] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:43.226223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f3227-9212-460a-a83b-5e398552be59 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.311038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c474c63-83d3-439f-bb36-5f00900d2752 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:43.332611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f3227-9212-460a-a83b-5e398552be59 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:43.348229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021013-0396-488a-95b3-e03aa1a3a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:43.348840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021013-0396-488a-95b3-e03aa1a3a44c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.349150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021013-0396-488a-95b3-e03aa1a3a44c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.349619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021013-0396-488a-95b3-e03aa1a3a44c None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.350373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021013-0396-488a-95b3-e03aa1a3a44c None admin] RBAC: Adding query filter params (name=member) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:43.350687 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021013-0396-488a-95b3-e03aa1a3a44c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:43.352075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021013-0396-488a-95b3-e03aa1a3a44c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:43.356736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:43.361117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 57/117] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:43.365737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be78ab-57e2-4be7-bf70-380d19b08d40 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.365737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be78ab-57e2-4be7-bf70-380d19b08d40 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.365737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be78ab-57e2-4be7-bf70-380d19b08d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.374399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a8de4-c6dc-4b27-b8a9-3942ece8e5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:43.375048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a8de4-c6dc-4b27-b8a9-3942ece8e5a4 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.375330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a8de4-c6dc-4b27-b8a9-3942ece8e5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.375604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a8de4-c6dc-4b27-b8a9-3942ece8e5a4 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.376981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a8de4-c6dc-4b27-b8a9-3942ece8e5a4 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:43.378501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a8de4-c6dc-4b27-b8a9-3942ece8e5a4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:43.380681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 61/118] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:43.382055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b6be78ab-57e2-4be7-bf70-380d19b08d40 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:43.383288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b6be78ab-57e2-4be7-bf70-380d19b08d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:43.387001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a8de4-c6dc-4b27-b8a9-3942ece8e5a4 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.452369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:43.457361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 58/119] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:43.466832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024fa98-8ec1-479f-a36a-841aebeeb9b1 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.469035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024fa98-8ec1-479f-a36a-841aebeeb9b1 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.469035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024fa98-8ec1-479f-a36a-841aebeeb9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.485588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9024fa98-8ec1-479f-a36a-841aebeeb9b1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:43.486855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9024fa98-8ec1-479f-a36a-841aebeeb9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:43.499503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a8de4-c6dc-4b27-b8a9-3942ece8e5a4 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:43.534720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce29c69-a3ff-42b4-b54e-14d3a54d131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:43.535354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29c69-a3ff-42b4-b54e-14d3a54d131f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.535642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29c69-a3ff-42b4-b54e-14d3a54d131f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.535922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce29c69-a3ff-42b4-b54e-14d3a54d131f None admin] PATH_INFO: `/v3/users/2a2e93b317db4de6850826f625ccc3db` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.537197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce29c69-a3ff-42b4-b54e-14d3a54d131f None admin] RBAC: Authorizing `identity:get_user(user_id=2a2e93b317db4de6850826f625ccc3db)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:43.538775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce29c69-a3ff-42b4-b54e-14d3a54d131f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:43.540481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 62/120] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/users/2a2e93b317db4de6850826f625ccc3db => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:43.541377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce29c69-a3ff-42b4-b54e-14d3a54d131f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.566493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 59/121] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:43.567608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024fa98-8ec1-479f-a36a-841aebeeb9b1 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.643195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:43.717112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce29c69-a3ff-42b4-b54e-14d3a54d131f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:43.763359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024fa98-8ec1-479f-a36a-841aebeeb9b1 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:43.776653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da5091-c997-4dd5-8799-13c1d0de3e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:43.777319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da5091-c997-4dd5-8799-13c1d0de3e4c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.777597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da5091-c997-4dd5-8799-13c1d0de3e4c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.777866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da5091-c997-4dd5-8799-13c1d0de3e4c None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.779091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da5091-c997-4dd5-8799-13c1d0de3e4c None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:43.780715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da5091-c997-4dd5-8799-13c1d0de3e4c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:43.782991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 63/122] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:43.783745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da5091-c997-4dd5-8799-13c1d0de3e4c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.801692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:43.808274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:43.808916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.809229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.810027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.810869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] RBAC: Adding query filter params (role.id=8041ef90cc68470fa80216023a3091c8, scope.project.id=b6271cacfed747da992ff32e5f4813f9, user.id=ecbecebf1fc5466393a2a83eb2488e94) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:43.811322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:43.812905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:43.819222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 60/123] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/role_assignments?user.id=ecbecebf1fc5466393a2a83eb2488e94&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=8041ef90cc68470fa80216023a3091c8 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:43.819969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.902063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da5091-c997-4dd5-8799-13c1d0de3e4c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:43.938780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:43.950076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989cbb-d9d8-4808-8a29-bc97ab57de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:43.950825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989cbb-d9d8-4808-8a29-bc97ab57de6a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:43.951172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989cbb-d9d8-4808-8a29-bc97ab57de6a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:43.951516 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989cbb-d9d8-4808-8a29-bc97ab57de6a None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:43.952756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c989cbb-d9d8-4808-8a29-bc97ab57de6a None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:43.954893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c989cbb-d9d8-4808-8a29-bc97ab57de6a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:43.957076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 64/124] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:43.958316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989cbb-d9d8-4808-8a29-bc97ab57de6a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:43.990090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c0c52-e052-4375-99a0-3cc7e4afe04a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.032804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f980d-742e-4c7f-839c-003fceb7b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.033529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f980d-742e-4c7f-839c-003fceb7b156 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.033905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f980d-742e-4c7f-839c-003fceb7b156 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.034421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f980d-742e-4c7f-839c-003fceb7b156 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.047002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f980d-742e-4c7f-839c-003fceb7b156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e51f980d-742e-4c7f-839c-003fceb7b156 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:44.053446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:44.054335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:44.055068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 61/125] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:44.055673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f980d-742e-4c7f-839c-003fceb7b156 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:44.097820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989cbb-d9d8-4808-8a29-bc97ab57de6a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.124251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:44.156731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a9693-b848-490a-b4e0-d0207a4b4697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.158132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a9693-b848-490a-b4e0-d0207a4b4697 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.158421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a9693-b848-490a-b4e0-d0207a4b4697 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.158698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a9693-b848-490a-b4e0-d0207a4b4697 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.160124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a9693-b848-490a-b4e0-d0207a4b4697 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:44.162273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a9693-b848-490a-b4e0-d0207a4b4697 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:44.165134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 65/126] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:43 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:44.165980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a9693-b848-490a-b4e0-d0207a4b4697 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:44.183342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f980d-742e-4c7f-839c-003fceb7b156 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.223932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334a9a5-d7a5-4315-a38b-80c04dd41ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.224695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334a9a5-d7a5-4315-a38b-80c04dd41ea0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.225246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334a9a5-d7a5-4315-a38b-80c04dd41ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.225525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334a9a5-d7a5-4315-a38b-80c04dd41ea0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.238394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334a9a5-d7a5-4315-a38b-80c04dd41ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e334a9a5-d7a5-4315-a38b-80c04dd41ea0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:44.246237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:44.247988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 62/127] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:34:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:44.249845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334a9a5-d7a5-4315-a38b-80c04dd41ea0 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:44.251564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:44.303352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a9693-b848-490a-b4e0-d0207a4b4697 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.349286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8519f-7db5-4f37-8528-2d515483f99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.353773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8519f-7db5-4f37-8528-2d515483f99b None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.354349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8519f-7db5-4f37-8528-2d515483f99b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.355115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8519f-7db5-4f37-8528-2d515483f99b None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9/users/2a2e93b317db4de6850826f625ccc3db/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.358262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8519f-7db5-4f37-8528-2d515483f99b None admin] RBAC: Authorizing `identity:create_grant(project_id=b6271cacfed747da992ff32e5f4813f9, user_id=2a2e93b317db4de6850826f625ccc3db, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:44.361156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8519f-7db5-4f37-8528-2d515483f99b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:44.365940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334a9a5-d7a5-4315-a38b-80c04dd41ea0 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.377200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 66/128] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:34:44 2022] PUT /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9/users/2a2e93b317db4de6850826f625ccc3db/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:34:44.382768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8519f-7db5-4f37-8528-2d515483f99b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:44.392923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:44.441515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.442191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.442471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.442741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.443420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] RBAC: Adding query filter params (name=member) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:44.443721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:44.448320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:44.471314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 63/129] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:44.472094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:44.565632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8519f-7db5-4f37-8528-2d515483f99b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.614315 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:44.626915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8160309-7ba0-4226-a2b7-f98241978259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.628110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8160309-7ba0-4226-a2b7-f98241978259 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.628624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8160309-7ba0-4226-a2b7-f98241978259 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.629484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8160309-7ba0-4226-a2b7-f98241978259 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9/users/9190860e1df04a81b08ae447688ed74d/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.631906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8160309-7ba0-4226-a2b7-f98241978259 None admin] RBAC: Authorizing `identity:create_grant(project_id=b6271cacfed747da992ff32e5f4813f9, user_id=9190860e1df04a81b08ae447688ed74d, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:44.633510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8160309-7ba0-4226-a2b7-f98241978259 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:44.656107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 67/130] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:34:44 2022] PUT /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9/users/9190860e1df04a81b08ae447688ed74d/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:34:44.657629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8160309-7ba0-4226-a2b7-f98241978259 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:44.667996 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b8165-d2a3-49f3-8e7a-35126bf12921 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.710267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d64a6d-597f-4ea7-8cf3-82a6c04920d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.711157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64a6d-597f-4ea7-8cf3-82a6c04920d1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.711583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64a6d-597f-4ea7-8cf3-82a6c04920d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.712035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d64a6d-597f-4ea7-8cf3-82a6c04920d1 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.712985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d64a6d-597f-4ea7-8cf3-82a6c04920d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:44.714188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d64a6d-597f-4ea7-8cf3-82a6c04920d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:44.715849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d64a6d-597f-4ea7-8cf3-82a6c04920d1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:44.725868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 64/131] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:34:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:44.731068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fc14-54a3-46e6-8607-df47fe698e58 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.731624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fc14-54a3-46e6-8607-df47fe698e58 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.732099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331fc14-54a3-46e6-8607-df47fe698e58 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.733638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 65/132] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:44.736798 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331fc14-54a3-46e6-8607-df47fe698e58 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:44.790279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:44.790949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8160309-7ba0-4226-a2b7-f98241978259 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.831645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983dd0a-97f7-409a-be8f-d2084fba3ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.832293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983dd0a-97f7-409a-be8f-d2084fba3ddc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.832578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983dd0a-97f7-409a-be8f-d2084fba3ddc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.832856 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983dd0a-97f7-409a-be8f-d2084fba3ddc None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.834036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983dd0a-97f7-409a-be8f-d2084fba3ddc None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:44.835637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983dd0a-97f7-409a-be8f-d2084fba3ddc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:44.837547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 68/133] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:44 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:44.838976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89920c4d-f61a-4f33-9a3c-26b0f96f273b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.839280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89920c4d-f61a-4f33-9a3c-26b0f96f273b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.839549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89920c4d-f61a-4f33-9a3c-26b0f96f273b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.863967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-89920c4d-f61a-4f33-9a3c-26b0f96f273b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:44.865273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-89920c4d-f61a-4f33-9a3c-26b0f96f273b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:44.865481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331fc14-54a3-46e6-8607-df47fe698e58 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:44.912081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82038891-5031-4a1a-b42a-8c6e645f4305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:44.912854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82038891-5031-4a1a-b42a-8c6e645f4305 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:44.913180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82038891-5031-4a1a-b42a-8c6e645f4305 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:44.913453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82038891-5031-4a1a-b42a-8c6e645f4305 None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:44.914675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82038891-5031-4a1a-b42a-8c6e645f4305 None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:44.916183 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82038891-5031-4a1a-b42a-8c6e645f4305 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:44.918765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 66/134] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:44 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:44.919624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82038891-5031-4a1a-b42a-8c6e645f4305 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:44.943714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:44.973884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 69/135] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:44.977988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89920c4d-f61a-4f33-9a3c-26b0f96f273b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:45.065751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82038891-5031-4a1a-b42a-8c6e645f4305 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:45.078467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:45.110193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bb3d9-2f88-4888-9a0e-d70b4c4fc741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:45.111217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb3d9-2f88-4888-9a0e-d70b4c4fc741 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:45.111677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb3d9-2f88-4888-9a0e-d70b4c4fc741 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:45.112101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb3d9-2f88-4888-9a0e-d70b4c4fc741 None admin] PATH_INFO: `/v3/projects/ada241b36a134695b8f5258a0c7af039` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:45.113756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bb3d9-2f88-4888-9a0e-d70b4c4fc741 None admin] RBAC: Authorizing `identity:get_project(project_id=ada241b36a134695b8f5258a0c7af039)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:45.115489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bb3d9-2f88-4888-9a0e-d70b4c4fc741 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:45.117674 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 67/136] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:44 2022] GET /identity/v3/projects/ada241b36a134695b8f5258a0c7af039 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:45.125549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bb3d9-2f88-4888-9a0e-d70b4c4fc741 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:45.177915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89920c4d-f61a-4f33-9a3c-26b0f96f273b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:45.249300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd36cb-6bd0-45f9-a8a7-9a16463b2092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:45.250975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd36cb-6bd0-45f9-a8a7-9a16463b2092 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:45.251272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd36cb-6bd0-45f9-a8a7-9a16463b2092 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:45.251551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd36cb-6bd0-45f9-a8a7-9a16463b2092 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:45.253898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd36cb-6bd0-45f9-a8a7-9a16463b2092 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:45.257818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd36cb-6bd0-45f9-a8a7-9a16463b2092 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:45.259642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 70/137] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:44 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:45.269806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd36cb-6bd0-45f9-a8a7-9a16463b2092 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:45.343023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bb3d9-2f88-4888-9a0e-d70b4c4fc741 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:45.393139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:45.438337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e2e2-f48b-406f-810a-4f6e712c3f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:45.439024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e2e2-f48b-406f-810a-4f6e712c3f64 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:45.439369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e2e2-f48b-406f-810a-4f6e712c3f64 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:45.439654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396e2e2-f48b-406f-810a-4f6e712c3f64 None admin] PATH_INFO: `/v3/projects/ada241b36a134695b8f5258a0c7af039/users/9190860e1df04a81b08ae447688ed74d/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:45.449831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e2e2-f48b-406f-810a-4f6e712c3f64 None admin] RBAC: Authorizing `identity:create_grant(project_id=ada241b36a134695b8f5258a0c7af039, user_id=9190860e1df04a81b08ae447688ed74d, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:45.451321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396e2e2-f48b-406f-810a-4f6e712c3f64 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:45.452071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd36cb-6bd0-45f9-a8a7-9a16463b2092 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:45.490621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cefe9-d71e-4c79-957d-4adfaba47cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:45.491297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cefe9-d71e-4c79-957d-4adfaba47cb2 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:45.491683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cefe9-d71e-4c79-957d-4adfaba47cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:45.492048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cefe9-d71e-4c79-957d-4adfaba47cb2 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9/users/9190860e1df04a81b08ae447688ed74d/roles/eb7bedf01e0744eeb26681978db4121c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:45.495646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cefe9-d71e-4c79-957d-4adfaba47cb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b6271cacfed747da992ff32e5f4813f9, user_id=9190860e1df04a81b08ae447688ed74d, role_id=eb7bedf01e0744eeb26681978db4121c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:45.497253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cefe9-d71e-4c79-957d-4adfaba47cb2 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:45.499421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 68/138] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:34:45 2022] PUT /identity/v3/projects/ada241b36a134695b8f5258a0c7af039/users/9190860e1df04a81b08ae447688ed74d/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:34:45.500055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e2e2-f48b-406f-810a-4f6e712c3f64 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:45.514690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 71/139] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:34:45 2022] PUT /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9/users/9190860e1df04a81b08ae447688ed74d/roles/eb7bedf01e0744eeb26681978db4121c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:34:45.551971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:45.637183 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396e2e2-f48b-406f-810a-4f6e712c3f64 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:45.678327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7aa6d-015b-4fc2-934f-6286b9c237f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:45.678902 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7aa6d-015b-4fc2-934f-6286b9c237f5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:45.679185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7aa6d-015b-4fc2-934f-6286b9c237f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:45.679460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7aa6d-015b-4fc2-934f-6286b9c237f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:45.680747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7aa6d-015b-4fc2-934f-6286b9c237f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:45.683230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7aa6d-015b-4fc2-934f-6286b9c237f5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:45.685580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 69/140] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:45.691799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cefe9-d71e-4c79-957d-4adfaba47cb2 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:45.852455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cefe9-d71e-4c79-957d-4adfaba47cb2 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:45.864100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:45.897367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf8907-38f1-404c-ac15-f39339a66616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:45.897997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf8907-38f1-404c-ac15-f39339a66616 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:45.898270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf8907-38f1-404c-ac15-f39339a66616 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:45.898538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf8907-38f1-404c-ac15-f39339a66616 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:45.899267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf8907-38f1-404c-ac15-f39339a66616 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:45.900709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf8907-38f1-404c-ac15-f39339a66616 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:45.936929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 72/141] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:34:45 2022] POST /identity/v3/users => generated 283 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:34:46.494455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f2bee-3e39-4a4a-b39a-2eef90b61077 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:46.495070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f2bee-3e39-4a4a-b39a-2eef90b61077 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:46.495070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f2bee-3e39-4a4a-b39a-2eef90b61077 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:46.496430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 70/142] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:46.501819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db27d1-e598-4dd2-ad90-454bdc42fe25 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:46.502211 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db27d1-e598-4dd2-ad90-454bdc42fe25 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:46.502387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db27d1-e598-4dd2-ad90-454bdc42fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:46.542128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-46db27d1-e598-4dd2-ad90-454bdc42fe25 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:46.543340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-46db27d1-e598-4dd2-ad90-454bdc42fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:46.623778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 73/143] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:47.117835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f2bee-3e39-4a4a-b39a-2eef90b61077 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:47.226047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0684464-89c0-4f66-b742-913391b65e5d None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:47.226709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0684464-89c0-4f66-b742-913391b65e5d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:47.227202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0684464-89c0-4f66-b742-913391b65e5d None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:47.228765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 74/144] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:47.234370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e97bb6-c828-484c-a6a8-9e589dbc8a82 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:47.234907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e97bb6-c828-484c-a6a8-9e589dbc8a82 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:47.235378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e97bb6-c828-484c-a6a8-9e589dbc8a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:47.246679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f2bee-3e39-4a4a-b39a-2eef90b61077 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:47.248358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:47.258117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-c7e97bb6-c828-484c-a6a8-9e589dbc8a82 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:47.259577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c7e97bb6-c828-484c-a6a8-9e589dbc8a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:47.323178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c2e46-19df-4e31-b9fa-e20652c63e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:47.324228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c2e46-19df-4e31-b9fa-e20652c63e5c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:47.324228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c2e46-19df-4e31-b9fa-e20652c63e5c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:47.324408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c2e46-19df-4e31-b9fa-e20652c63e5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:47.343861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 75/145] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:47.358046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c2e46-19df-4e31-b9fa-e20652c63e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-ef1c2e46-19df-4e31-b9fa-e20652c63e5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:47.362986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:47.363939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:47.365170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 71/146] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:47.370639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e97bb6-c828-484c-a6a8-9e589dbc8a82 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:47.408406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:47.508374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e97bb6-c828-484c-a6a8-9e589dbc8a82 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:47.551521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:47.552134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:47.552411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:47.552680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:47.553394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:47.553693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:47.555083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:47.566264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 76/147] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:47.572021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c2e46-19df-4e31-b9fa-e20652c63e5c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:47.680599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c2e46-19df-4e31-b9fa-e20652c63e5c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:47.716828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d71a31-93c2-43f2-9247-de0958bb872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:47.717509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71a31-93c2-43f2-9247-de0958bb872c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:47.717790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71a31-93c2-43f2-9247-de0958bb872c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:47.718063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71a31-93c2-43f2-9247-de0958bb872c None admin] PATH_INFO: `/v3/users/ecbecebf1fc5466393a2a83eb2488e94` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:47.719194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d71a31-93c2-43f2-9247-de0958bb872c None admin] RBAC: Authorizing `identity:get_user(user_id=ecbecebf1fc5466393a2a83eb2488e94)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:47.720635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d71a31-93c2-43f2-9247-de0958bb872c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:47.725889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 72/148] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:47 2022] GET /identity/v3/users/ecbecebf1fc5466393a2a83eb2488e94 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:47.728304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:47.836216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:47.851687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb64d0c-a4d4-4dbc-abbb-7dc441894535 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:47.898158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f8817-c15f-48c1-a845-5994481b0253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:47.898861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8817-c15f-48c1-a845-5994481b0253 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:47.899142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8817-c15f-48c1-a845-5994481b0253 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:47.899412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8817-c15f-48c1-a845-5994481b0253 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:47.905246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f8817-c15f-48c1-a845-5994481b0253 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:47.906706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f8817-c15f-48c1-a845-5994481b0253 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:47.908370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 77/149] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:47 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:47.914458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d71a31-93c2-43f2-9247-de0958bb872c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:47.952732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:47.969805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f8817-c15f-48c1-a845-5994481b0253 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:48.104826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d71a31-93c2-43f2-9247-de0958bb872c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:48.120837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f8817-c15f-48c1-a845-5994481b0253 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:48.159134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea769dd-1990-4606-ac98-e6fb04201b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:48.159771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea769dd-1990-4606-ac98-e6fb04201b74 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:48.160049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea769dd-1990-4606-ac98-e6fb04201b74 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:48.160318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea769dd-1990-4606-ac98-e6fb04201b74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:48.172283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea769dd-1990-4606-ac98-e6fb04201b74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-3ea769dd-1990-4606-ac98-e6fb04201b74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:48.178449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:48.179273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:48.180009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 78/150] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:34:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:34:48.187456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea769dd-1990-4606-ac98-e6fb04201b74 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:48.202352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dceed-2d31-4ea3-b43b-0cacfd38ff5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:48.203041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dceed-2d31-4ea3-b43b-0cacfd38ff5f None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:48.203327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dceed-2d31-4ea3-b43b-0cacfd38ff5f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:48.203599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dceed-2d31-4ea3-b43b-0cacfd38ff5f None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9/users/ecbecebf1fc5466393a2a83eb2488e94/roles/8041ef90cc68470fa80216023a3091c8` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:48.205604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dceed-2d31-4ea3-b43b-0cacfd38ff5f None admin] RBAC: Authorizing `identity:create_grant(project_id=b6271cacfed747da992ff32e5f4813f9, user_id=ecbecebf1fc5466393a2a83eb2488e94, role_id=8041ef90cc68470fa80216023a3091c8)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:48.207044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dceed-2d31-4ea3-b43b-0cacfd38ff5f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:48.207492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:48.260136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 73/151] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:34:47 2022] PUT /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9/users/ecbecebf1fc5466393a2a83eb2488e94/roles/8041ef90cc68470fa80216023a3091c8 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:34:48.338935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea769dd-1990-4606-ac98-e6fb04201b74 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:48.384376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:48.385327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:48.385855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:48.386366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:48.387241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:48.387733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:48.389343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:48.404121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 79/152] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:48.410255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dceed-2d31-4ea3-b43b-0cacfd38ff5f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:48.567832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:48.579195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dceed-2d31-4ea3-b43b-0cacfd38ff5f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:48.619921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cec57-ace5-4707-a077-d1feaefdc73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:48.620565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cec57-ace5-4707-a077-d1feaefdc73a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:48.620853 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cec57-ace5-4707-a077-d1feaefdc73a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:48.621142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cec57-ace5-4707-a077-d1feaefdc73a None admin] PATH_INFO: `/v3/users/2a2e93b317db4de6850826f625ccc3db` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:48.622279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cec57-ace5-4707-a077-d1feaefdc73a None admin] RBAC: Authorizing `identity:get_user(user_id=2a2e93b317db4de6850826f625ccc3db)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:48.623737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cec57-ace5-4707-a077-d1feaefdc73a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:48.625667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 74/153] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:48 2022] GET /identity/v3/users/2a2e93b317db4de6850826f625ccc3db => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:48.632008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:48.752471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:48.785917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550207d-4588-4c6a-ad8e-15b68d8e048d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:48.828358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639f90-6efc-4b0d-92db-92d6eb56accb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:48.829395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639f90-6efc-4b0d-92db-92d6eb56accb None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:48.829873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639f90-6efc-4b0d-92db-92d6eb56accb None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:48.830355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639f90-6efc-4b0d-92db-92d6eb56accb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:48.903763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a639f90-6efc-4b0d-92db-92d6eb56accb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:48.905429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a639f90-6efc-4b0d-92db-92d6eb56accb None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:48.988337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:49.016543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 80/154] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:49.021254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cec57-ace5-4707-a077-d1feaefdc73a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:49.021840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df3a12-b19f-47de-8b02-1b8cd46d5061 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:49.022186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df3a12-b19f-47de-8b02-1b8cd46d5061 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:49.022465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df3a12-b19f-47de-8b02-1b8cd46d5061 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:49.024098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 81/155] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:34:49.031302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a647af-f943-4a6b-b7e3-7f637390d3e8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:49.031584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a647af-f943-4a6b-b7e3-7f637390d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:49.031854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a647af-f943-4a6b-b7e3-7f637390d3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:49.054152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-88a647af-f943-4a6b-b7e3-7f637390d3e8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:49.055501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-88a647af-f943-4a6b-b7e3-7f637390d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:49.151167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 82/156] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:34:49.228582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cec57-ace5-4707-a077-d1feaefdc73a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:49.286076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d746f-0182-4ded-a530-7dfc734a7e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:49.287115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d746f-0182-4ded-a530-7dfc734a7e85 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:49.287592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d746f-0182-4ded-a530-7dfc734a7e85 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:49.288060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d746f-0182-4ded-a530-7dfc734a7e85 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:49.289582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d746f-0182-4ded-a530-7dfc734a7e85 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:49.291341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d746f-0182-4ded-a530-7dfc734a7e85 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:49.294246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 75/157] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:49 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:49.300058 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a647af-f943-4a6b-b7e3-7f637390d3e8 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:49.469557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a647af-f943-4a6b-b7e3-7f637390d3e8 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:49.487061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:49.515605 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9511dd1-9a7a-4813-bafe-56f68af702e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:49.516206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9511dd1-9a7a-4813-bafe-56f68af702e7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:49.516478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9511dd1-9a7a-4813-bafe-56f68af702e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:49.516744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9511dd1-9a7a-4813-bafe-56f68af702e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:49.517494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9511dd1-9a7a-4813-bafe-56f68af702e7 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=b6271cacfed747da992ff32e5f4813f9, user.id=2a2e93b317db4de6850826f625ccc3db) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:49.517797 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9511dd1-9a7a-4813-bafe-56f68af702e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:49.519223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9511dd1-9a7a-4813-bafe-56f68af702e7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:49.530968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 83/158] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:49 2022] GET /identity/v3/role_assignments?user.id=2a2e93b317db4de6850826f625ccc3db&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=b131b61299154544a627315649090a49 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:49.616572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1183c0-5e27-4362-96e7-522ab64e501b None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:49.617569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1183c0-5e27-4362-96e7-522ab64e501b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:49.618057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1183c0-5e27-4362-96e7-522ab64e501b None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:49.619718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 76/159] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:49.625275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c3f9-d1c3-4ad3-b26a-237dcbf0aaac None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:49.625563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c3f9-d1c3-4ad3-b26a-237dcbf0aaac None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:49.625833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c3f9-d1c3-4ad3-b26a-237dcbf0aaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:49.654070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f004c3f9-d1c3-4ad3-b26a-237dcbf0aaac None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:49.655258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f004c3f9-d1c3-4ad3-b26a-237dcbf0aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:49.748816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 84/160] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:49.839253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1183c0-5e27-4362-96e7-522ab64e501b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:49.879000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:49.984810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1183c0-5e27-4362-96e7-522ab64e501b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:50.033410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260eee77-db1e-4971-a6d0-1d06ea47c203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:50.034060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eee77-db1e-4971-a6d0-1d06ea47c203 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:50.034341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eee77-db1e-4971-a6d0-1d06ea47c203 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:50.034611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260eee77-db1e-4971-a6d0-1d06ea47c203 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:50.053304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260eee77-db1e-4971-a6d0-1d06ea47c203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-260eee77-db1e-4971-a6d0-1d06ea47c203 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:50.058690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:50.059790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:50.060739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:50.063023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 77/161] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:50.068961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004c3f9-d1c3-4ad3-b26a-237dcbf0aaac None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:50.128826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e6ac2-c228-4059-91ee-5e4b800bd154 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:50.129180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e6ac2-c228-4059-91ee-5e4b800bd154 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:50.129403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e6ac2-c228-4059-91ee-5e4b800bd154 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:50.131558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 78/162] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:34:50.138131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e262-31b9-49bd-9e33-b2db3c3fc08c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:50.138403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e262-31b9-49bd-9e33-b2db3c3fc08c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:50.138661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5e262-31b9-49bd-9e33-b2db3c3fc08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:50.170101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ded5e262-31b9-49bd-9e33-b2db3c3fc08c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:50.171303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ded5e262-31b9-49bd-9e33-b2db3c3fc08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:50.248377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004c3f9-d1c3-4ad3-b26a-237dcbf0aaac None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:50.250076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:50.263056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 79/163] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:50.300494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:50.301137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:50.301420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:50.301689 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:50.302356 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:50.302656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:50.304021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:50.320173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 85/164] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:50.326147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5e262-31b9-49bd-9e33-b2db3c3fc08c None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:50.354302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:50.424507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:50.451415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5e262-31b9-49bd-9e33-b2db3c3fc08c None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:50.518861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0aeed-f407-45db-9aa1-740487bdbcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:50.519530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0aeed-f407-45db-9aa1-740487bdbcb0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:50.519805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0aeed-f407-45db-9aa1-740487bdbcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:50.520073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0aeed-f407-45db-9aa1-740487bdbcb0 None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:50.521345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0aeed-f407-45db-9aa1-740487bdbcb0 None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:50.522852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0aeed-f407-45db-9aa1-740487bdbcb0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:50.529256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 80/165] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:50 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:50.531094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0aeed-f407-45db-9aa1-740487bdbcb0 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:50.634971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3badb9dc-778e-4308-a16f-f74b0c8eb1ad None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:50.635497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:50.653822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0aeed-f407-45db-9aa1-740487bdbcb0 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:50.693458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5433d-3d9e-462c-ba1f-9ce926291fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:50.695956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5433d-3d9e-462c-ba1f-9ce926291fb2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:50.696381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5433d-3d9e-462c-ba1f-9ce926291fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:50.697542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5433d-3d9e-462c-ba1f-9ce926291fb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:50.704226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df873e-b256-4eac-852f-748c47860870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:50.705591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df873e-b256-4eac-852f-748c47860870 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:50.707260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df873e-b256-4eac-852f-748c47860870 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:50.708922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df873e-b256-4eac-852f-748c47860870 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:50.744790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df873e-b256-4eac-852f-748c47860870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-02df873e-b256-4eac-852f-748c47860870 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:50.758716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:50.759884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:50.761655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:50.765482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 86/166] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:50.784819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df873e-b256-4eac-852f-748c47860870 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:50.829431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:50.854368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5433d-3d9e-462c-ba1f-9ce926291fb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:50.855902 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5433d-3d9e-462c-ba1f-9ce926291fb2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:50.940926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df873e-b256-4eac-852f-748c47860870 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:50.981374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886b97c-b739-4521-8875-f709bb6941bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:50.981928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b97c-b739-4521-8875-f709bb6941bd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:50.982208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b97c-b739-4521-8875-f709bb6941bd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:50.982484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886b97c-b739-4521-8875-f709bb6941bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:50.994039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886b97c-b739-4521-8875-f709bb6941bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-4886b97c-b739-4521-8875-f709bb6941bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:51.000752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:51.001654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:51.002430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 87/167] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:34:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:51.003511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886b97c-b739-4521-8875-f709bb6941bd None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:51.047095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:51.095930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 81/168] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 01:34:51.099813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5589d8-ec95-43c9-a2ee-4d85d8b955c9 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.100126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5589d8-ec95-43c9-a2ee-4d85d8b955c9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.100403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5589d8-ec95-43c9-a2ee-4d85d8b955c9 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.102592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 82/169] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:51.104245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5589d8-ec95-43c9-a2ee-4d85d8b955c9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:51.129272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:51.191264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886b97c-b739-4521-8875-f709bb6941bd None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:51.215176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5589d8-ec95-43c9-a2ee-4d85d8b955c9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:51.250887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:51.251422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.251702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.251970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.252632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:51.252934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:51.254388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:51.263255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 83/170] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:34:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:51.264366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:51.268911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65775-5ffe-4cb2-9bf1-dab8a5995918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:51.269765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65775-5ffe-4cb2-9bf1-dab8a5995918 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.270221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65775-5ffe-4cb2-9bf1-dab8a5995918 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.270671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65775-5ffe-4cb2-9bf1-dab8a5995918 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.271531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d65775-5ffe-4cb2-9bf1-dab8a5995918 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:51.272019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d65775-5ffe-4cb2-9bf1-dab8a5995918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:51.273602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d65775-5ffe-4cb2-9bf1-dab8a5995918 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:51.291176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 88/171] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:51.293201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d178c-2b1d-42aa-a85c-8ea7e40669e7 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.293687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d178c-2b1d-42aa-a85c-8ea7e40669e7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.294133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d178c-2b1d-42aa-a85c-8ea7e40669e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.314462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:51.334501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4a0d178c-2b1d-42aa-a85c-8ea7e40669e7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:51.336036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4a0d178c-2b1d-42aa-a85c-8ea7e40669e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:51.372955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78acf965-9b86-439a-b239-b3bda076fe8f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:51.409211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb31735-4c45-4d18-8424-3212f76381f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:51.410675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb31735-4c45-4d18-8424-3212f76381f1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.411163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb31735-4c45-4d18-8424-3212f76381f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.411619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb31735-4c45-4d18-8424-3212f76381f1 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.413248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb31735-4c45-4d18-8424-3212f76381f1 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:51.415479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb31735-4c45-4d18-8424-3212f76381f1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:51.417657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 84/172] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:51 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:51.418986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb31735-4c45-4d18-8424-3212f76381f1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:51.542845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb31735-4c45-4d18-8424-3212f76381f1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:51.546402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 89/173] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:51.547284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d178c-2b1d-42aa-a85c-8ea7e40669e7 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:51.562695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:51.595027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a9c7e-9754-403d-8d20-bfb530fdca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:51.596057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a9c7e-9754-403d-8d20-bfb530fdca0a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.596533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a9c7e-9754-403d-8d20-bfb530fdca0a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.597001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a9c7e-9754-403d-8d20-bfb530fdca0a None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.598437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a9c7e-9754-403d-8d20-bfb530fdca0a None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:51.600693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a9c7e-9754-403d-8d20-bfb530fdca0a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:51.603467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 85/174] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:51 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:51.604954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a9c7e-9754-403d-8d20-bfb530fdca0a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:51.639480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:51.664604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d178c-2b1d-42aa-a85c-8ea7e40669e7 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:51.715222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4152176-fbc4-4265-836a-ded2fdcfbad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:51.716000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4152176-fbc4-4265-836a-ded2fdcfbad7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.716223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4152176-fbc4-4265-836a-ded2fdcfbad7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.716604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4152176-fbc4-4265-836a-ded2fdcfbad7 None admin] PATH_INFO: `/v3/users/9190860e1df04a81b08ae447688ed74d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.717902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4152176-fbc4-4265-836a-ded2fdcfbad7 None admin] RBAC: Authorizing `identity:get_user(user_id=9190860e1df04a81b08ae447688ed74d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:51.719496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4152176-fbc4-4265-836a-ded2fdcfbad7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:51.721479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 90/175] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:51 2022] GET /identity/v3/users/9190860e1df04a81b08ae447688ed74d => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:51.722458 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204496c-0857-426c-8d14-9af589411b26 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.722740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204496c-0857-426c-8d14-9af589411b26 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.723013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204496c-0857-426c-8d14-9af589411b26 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.724217 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 91/176] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:51.729690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204496c-0857-426c-8d14-9af589411b26 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:51.750610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a9c7e-9754-403d-8d20-bfb530fdca0a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:51.794910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4d193-db8b-4dfb-9d89-4d2a79d1ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:51.795560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4d193-db8b-4dfb-9d89-4d2a79d1ec10 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.795842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4d193-db8b-4dfb-9d89-4d2a79d1ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.796114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4d193-db8b-4dfb-9d89-4d2a79d1ec10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.797005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4d193-db8b-4dfb-9d89-4d2a79d1ec10 None admin] RBAC: Adding query filter params (role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=b6271cacfed747da992ff32e5f4813f9, user.id=9190860e1df04a81b08ae447688ed74d) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:51.797339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4d193-db8b-4dfb-9d89-4d2a79d1ec10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:51.799144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4d193-db8b-4dfb-9d89-4d2a79d1ec10 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:51.807177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 86/177] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:51 2022] GET /identity/v3/role_assignments?user.id=9190860e1df04a81b08ae447688ed74d&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=18bebd2151824cdfa386be482ff8114b => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:51.808633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e50b8-496c-4802-bc65-015ac9234259 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.808957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e50b8-496c-4802-bc65-015ac9234259 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.809262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e50b8-496c-4802-bc65-015ac9234259 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.810692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 87/178] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:51.811431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e50b8-496c-4802-bc65-015ac9234259 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:51.877862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204496c-0857-426c-8d14-9af589411b26 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:51.903706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:51.923347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be727b0-ffc7-4d43-9adb-c8fb71fcc999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:51.923973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be727b0-ffc7-4d43-9adb-c8fb71fcc999 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:51.924249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be727b0-ffc7-4d43-9adb-c8fb71fcc999 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:51.924520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be727b0-ffc7-4d43-9adb-c8fb71fcc999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:51.941298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:51.973842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e50b8-496c-4802-bc65-015ac9234259 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.012063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be727b0-ffc7-4d43-9adb-c8fb71fcc999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:52.014530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be727b0-ffc7-4d43-9adb-c8fb71fcc999 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:52.019565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c022fc-7c5c-4ff6-a442-05795bdb3bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:52.020191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c022fc-7c5c-4ff6-a442-05795bdb3bf7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.020469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c022fc-7c5c-4ff6-a442-05795bdb3bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.020792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c022fc-7c5c-4ff6-a442-05795bdb3bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.116060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c022fc-7c5c-4ff6-a442-05795bdb3bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:52.117815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c022fc-7c5c-4ff6-a442-05795bdb3bf7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:52.149401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 92/179] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:52.149614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942775a-bd0e-41fb-bf4f-8424d40e204c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.149881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942775a-bd0e-41fb-bf4f-8424d40e204c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.150082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6942775a-bd0e-41fb-bf4f-8424d40e204c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.168919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6942775a-bd0e-41fb-bf4f-8424d40e204c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:52.170131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6942775a-bd0e-41fb-bf4f-8424d40e204c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:52.173723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:52.208676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:52.229978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 88/180] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:52.231220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a25527-f008-4540-a441-d638f0b2dd2a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.231505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a25527-f008-4540-a441-d638f0b2dd2a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.231781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a25527-f008-4540-a441-d638f0b2dd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.236493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 93/181] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:52.237349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942775a-bd0e-41fb-bf4f-8424d40e204c None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:52.249226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b0a25527-f008-4540-a441-d638f0b2dd2a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:52.250503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b0a25527-f008-4540-a441-d638f0b2dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:52.319190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 89/182] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:52.320056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a25527-f008-4540-a441-d638f0b2dd2a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:52.366869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6942775a-bd0e-41fb-bf4f-8424d40e204c None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.414085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b465f-a67a-46b9-a6ae-67e1e6ee79ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:52.414798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b465f-a67a-46b9-a6ae-67e1e6ee79ca None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.415076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b465f-a67a-46b9-a6ae-67e1e6ee79ca None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.415346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b465f-a67a-46b9-a6ae-67e1e6ee79ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.419661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:52.430400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b465f-a67a-46b9-a6ae-67e1e6ee79ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-d52b465f-a67a-46b9-a6ae-67e1e6ee79ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:52.436882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:52.437918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:52.438687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 94/183] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:34:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:52.439328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e5c5a-f36a-44b0-a5e7-940e9bc70e1e None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.439392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e5c5a-f36a-44b0-a5e7-940e9bc70e1e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.440190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e5c5a-f36a-44b0-a5e7-940e9bc70e1e None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.441323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 95/184] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:52.445546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e5c5a-f36a-44b0-a5e7-940e9bc70e1e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:52.454361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a25527-f008-4540-a441-d638f0b2dd2a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.481439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:52.496092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd65df-37a8-419c-92db-7c3f039e0d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:52.497002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd65df-37a8-419c-92db-7c3f039e0d60 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.497673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd65df-37a8-419c-92db-7c3f039e0d60 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.498234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd65df-37a8-419c-92db-7c3f039e0d60 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.499615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd65df-37a8-419c-92db-7c3f039e0d60 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:52.501413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd65df-37a8-419c-92db-7c3f039e0d60 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:52.503355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 90/185] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:52 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:52.504600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd65df-37a8-419c-92db-7c3f039e0d60 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:52.550485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e5c5a-f36a-44b0-a5e7-940e9bc70e1e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.584126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795eb2e-f2ef-46e6-bbf6-7eefb85769b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:52.584781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795eb2e-f2ef-46e6-bbf6-7eefb85769b2 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.585081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795eb2e-f2ef-46e6-bbf6-7eefb85769b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.585349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795eb2e-f2ef-46e6-bbf6-7eefb85769b2 None admin] PATH_INFO: `/v3/projects/ada241b36a134695b8f5258a0c7af039` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.586469 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795eb2e-f2ef-46e6-bbf6-7eefb85769b2 None admin] RBAC: Authorizing `identity:get_project(project_id=ada241b36a134695b8f5258a0c7af039)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:52.587916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795eb2e-f2ef-46e6-bbf6-7eefb85769b2 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:52.590412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 96/186] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:52 2022] GET /identity/v3/projects/ada241b36a134695b8f5258a0c7af039 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:52.591511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7202ea2-8394-401a-9db6-bc8c00ee28b8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.591786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7202ea2-8394-401a-9db6-bc8c00ee28b8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.592053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7202ea2-8394-401a-9db6-bc8c00ee28b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.609766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e7202ea2-8394-401a-9db6-bc8c00ee28b8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:52.610564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd65df-37a8-419c-92db-7c3f039e0d60 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.610944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e7202ea2-8394-401a-9db6-bc8c00ee28b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:52.644328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:52.644858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.645164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.645429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.646091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:52.646385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:52.647764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:52.656054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 91/187] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:52.656820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:52.665105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:52.678781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 97/188] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:52.679621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7202ea2-8394-401a-9db6-bc8c00ee28b8 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:52.710320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:52.767726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c6a74-749b-485b-8b02-91bbbb51b33d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.787074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7202ea2-8394-401a-9db6-bc8c00ee28b8 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.805054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be5a7-3b51-43c5-9063-69f4167a80e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:52.805714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be5a7-3b51-43c5-9063-69f4167a80e9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.805996 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be5a7-3b51-43c5-9063-69f4167a80e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.806270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be5a7-3b51-43c5-9063-69f4167a80e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.816883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5be5a7-3b51-43c5-9063-69f4167a80e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-ef5be5a7-3b51-43c5-9063-69f4167a80e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:52.822289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:52.823151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:52.823911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 92/189] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:34:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:52.824648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be5a7-3b51-43c5-9063-69f4167a80e9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:52.829700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:52.830315 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:52.830597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:52.830870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:52.831601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] RBAC: Adding query filter params (role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=b6271cacfed747da992ff32e5f4813f9, user.id=9190860e1df04a81b08ae447688ed74d) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:52.831911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:52.833365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:52.850065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 98/190] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:52 2022] GET /identity/v3/role_assignments?user.id=9190860e1df04a81b08ae447688ed74d&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=eb7bedf01e0744eeb26681978db4121c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:52.850065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:52.928789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:52.958771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be5a7-3b51-43c5-9063-69f4167a80e9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.986859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:52.993293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84baed-745b-47cf-b282-84a3d8f77ee5 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:52.999572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.000194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.000464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.000732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.001500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] RBAC: Adding query filter params (role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=ada241b36a134695b8f5258a0c7af039, user.id=9190860e1df04a81b08ae447688ed74d) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:53.001806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.003844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.012225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 93/191] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:52 2022] GET /identity/v3/role_assignments?user.id=9190860e1df04a81b08ae447688ed74d&scope.project.id=ada241b36a134695b8f5258a0c7af039&role.id=18bebd2151824cdfa386be482ff8114b => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:53.013090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.044168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0456bbd-6564-4b48-ad8a-6551ab1146d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.045286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0456bbd-6564-4b48-ad8a-6551ab1146d0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.045575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0456bbd-6564-4b48-ad8a-6551ab1146d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.045966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0456bbd-6564-4b48-ad8a-6551ab1146d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.047196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0456bbd-6564-4b48-ad8a-6551ab1146d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.048691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0456bbd-6564-4b48-ad8a-6551ab1146d0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.050601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 99/192] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:53.051265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0456bbd-6564-4b48-ad8a-6551ab1146d0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.158296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6dcb9-41ea-47d1-b8f8-ecfdc2ae632f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.199313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:53.201053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0456bbd-6564-4b48-ad8a-6551ab1146d0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.209726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585d9bf-cc0f-43bd-a03e-7044ecfc4930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.211039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585d9bf-cc0f-43bd-a03e-7044ecfc4930 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.211384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585d9bf-cc0f-43bd-a03e-7044ecfc4930 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.211717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585d9bf-cc0f-43bd-a03e-7044ecfc4930 None admin] PATH_INFO: `/v3/users/3d536721c74b47c6adbcee5d5db41557` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.232291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585d9bf-cc0f-43bd-a03e-7044ecfc4930 None admin] RBAC: Authorizing `identity:get_user(user_id=3d536721c74b47c6adbcee5d5db41557)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.234152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585d9bf-cc0f-43bd-a03e-7044ecfc4930 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.235973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 94/193] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/users/3d536721c74b47c6adbcee5d5db41557 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:53.236884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585d9bf-cc0f-43bd-a03e-7044ecfc4930 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.237407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.238013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.238297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.238569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.239890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:53.241477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.242548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.251421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 100/194] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:53.252334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.285100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:53.347187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585d9bf-cc0f-43bd-a03e-7044ecfc4930 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.359204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624331e-1b67-4de6-a299-52ec64c4ffd6 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.380467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92521b5-ecb2-423f-a516-ef5065bfcdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.381128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92521b5-ecb2-423f-a516-ef5065bfcdaa None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.381406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92521b5-ecb2-423f-a516-ef5065bfcdaa None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.381681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92521b5-ecb2-423f-a516-ef5065bfcdaa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.392110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92521b5-ecb2-423f-a516-ef5065bfcdaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-d92521b5-ecb2-423f-a516-ef5065bfcdaa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:53.397304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:53.398365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 95/195] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:53.399310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92521b5-ecb2-423f-a516-ef5065bfcdaa None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.404854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9775812-dadb-4fc8-bdf8-8b99332ae1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.405504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775812-dadb-4fc8-bdf8-8b99332ae1c0 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.405779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775812-dadb-4fc8-bdf8-8b99332ae1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.406045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9775812-dadb-4fc8-bdf8-8b99332ae1c0 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.406693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9775812-dadb-4fc8-bdf8-8b99332ae1c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.408139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9775812-dadb-4fc8-bdf8-8b99332ae1c0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.429007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 101/196] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:34:53 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:34:53.429799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9775812-dadb-4fc8-bdf8-8b99332ae1c0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.460582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:53.522854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92521b5-ecb2-423f-a516-ef5065bfcdaa None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.531360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:53.555481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9775812-dadb-4fc8-bdf8-8b99332ae1c0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.561710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb422c-1524-4d03-8ce3-69356c31d92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.562772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb422c-1524-4d03-8ce3-69356c31d92a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.563255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb422c-1524-4d03-8ce3-69356c31d92a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.563746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb422c-1524-4d03-8ce3-69356c31d92a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.596136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4162052-6034-494a-9e55-77755f93693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.596819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4162052-6034-494a-9e55-77755f93693f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.597130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4162052-6034-494a-9e55-77755f93693f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.597630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4162052-6034-494a-9e55-77755f93693f None admin] PATH_INFO: `/v3/users/3d536721c74b47c6adbcee5d5db41557` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.598556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4162052-6034-494a-9e55-77755f93693f None admin] RBAC: Authorizing `identity:get_user(user_id=3d536721c74b47c6adbcee5d5db41557)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.600026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4162052-6034-494a-9e55-77755f93693f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.601944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 102/197] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/users/3d536721c74b47c6adbcee5d5db41557 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:53.602799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4162052-6034-494a-9e55-77755f93693f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.648199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb422c-1524-4d03-8ce3-69356c31d92a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.650150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb422c-1524-4d03-8ce3-69356c31d92a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.711211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:53.719776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4162052-6034-494a-9e55-77755f93693f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.762780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.763689 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.763963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.764237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.764887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] RBAC: Adding query filter params (name=reader) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:53.765248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.766626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.774853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 103/198] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:53.775664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.786151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 96/199] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:53.786830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb422c-1524-4d03-8ce3-69356c31d92a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.796877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:53.894783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61e581-6a98-47e8-87f5-e866d17c889b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.912129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb422c-1524-4d03-8ce3-69356c31d92a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:53.938820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83385afe-60ab-4acb-8b6a-31afd40ec063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.940353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83385afe-60ab-4acb-8b6a-31afd40ec063 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.940887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83385afe-60ab-4acb-8b6a-31afd40ec063 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.941583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83385afe-60ab-4acb-8b6a-31afd40ec063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.956254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8123206-9b3c-4dde-bb1f-0fcf3ce88ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:53.957508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123206-9b3c-4dde-bb1f-0fcf3ce88ba3 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:53.957883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123206-9b3c-4dde-bb1f-0fcf3ce88ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:53.958251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123206-9b3c-4dde-bb1f-0fcf3ce88ba3 None admin] PATH_INFO: `/v3/users/ecbecebf1fc5466393a2a83eb2488e94` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:53.960381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8123206-9b3c-4dde-bb1f-0fcf3ce88ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=ecbecebf1fc5466393a2a83eb2488e94)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:53.962237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8123206-9b3c-4dde-bb1f-0fcf3ce88ba3 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:53.964768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 97/200] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/users/ecbecebf1fc5466393a2a83eb2488e94 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:53.965507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8123206-9b3c-4dde-bb1f-0fcf3ce88ba3 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:53.989271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:54.039591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83385afe-60ab-4acb-8b6a-31afd40ec063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:54.042091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83385afe-60ab-4acb-8b6a-31afd40ec063 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:54.061048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:54.093858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8123206-9b3c-4dde-bb1f-0fcf3ce88ba3 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:54.134024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc34d3-e8ed-4d3e-8c87-031141a21bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:54.134799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc34d3-e8ed-4d3e-8c87-031141a21bae None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:54.135323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc34d3-e8ed-4d3e-8c87-031141a21bae None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:54.135660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc34d3-e8ed-4d3e-8c87-031141a21bae None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:54.136851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc34d3-e8ed-4d3e-8c87-031141a21bae None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:54.139561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc34d3-e8ed-4d3e-8c87-031141a21bae None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:54.145199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 98/201] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:54.147494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc34d3-e8ed-4d3e-8c87-031141a21bae None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:54.173916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 104/202] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:54.175137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83385afe-60ab-4acb-8b6a-31afd40ec063 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:54.234954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:54.252046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc34d3-e8ed-4d3e-8c87-031141a21bae None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:54.287635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d522e-81a4-4715-a007-33201976f217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:54.288251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d522e-81a4-4715-a007-33201976f217 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:54.288526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d522e-81a4-4715-a007-33201976f217 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:54.288794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d522e-81a4-4715-a007-33201976f217 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:54.289207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83385afe-60ab-4acb-8b6a-31afd40ec063 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:54.305409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:54.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:54.330744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:54.331117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:54.331731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:54.333486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=5a9037a124a2489992eba863bf5e9b44, user.id=3d536721c74b47c6adbcee5d5db41557) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:54.333924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:54.335472 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:54.349105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 105/203] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:54 2022] GET /identity/v3/role_assignments?user.id=3d536721c74b47c6adbcee5d5db41557&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=b131b61299154544a627315649090a49 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:54.349105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:54.405521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d522e-81a4-4715-a007-33201976f217 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:54.407267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d522e-81a4-4715-a007-33201976f217 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:54.560167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242c497-5bd2-4f4f-8423-0089a686d033 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:54.578689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:54.601009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d760056-c5a9-4aa4-ad06-905c97a2d1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:54.601697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d760056-c5a9-4aa4-ad06-905c97a2d1aa None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:54.601975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d760056-c5a9-4aa4-ad06-905c97a2d1aa None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:54.602247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d760056-c5a9-4aa4-ad06-905c97a2d1aa None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:54.603814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 99/204] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:54.604911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d760056-c5a9-4aa4-ad06-905c97a2d1aa None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:54.607029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d760056-c5a9-4aa4-ad06-905c97a2d1aa None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:54.608961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 106/205] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:54 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:54.609531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d522e-81a4-4715-a007-33201976f217 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:54.615470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d760056-c5a9-4aa4-ad06-905c97a2d1aa None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:54.694960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:54.728218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d522e-81a4-4715-a007-33201976f217 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:54.774713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7580b-c9cc-4c7c-8bf5-d38705d64041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:54.775720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7580b-c9cc-4c7c-8bf5-d38705d64041 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:54.776783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7580b-c9cc-4c7c-8bf5-d38705d64041 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:54.777276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7580b-c9cc-4c7c-8bf5-d38705d64041 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:54.778747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7580b-c9cc-4c7c-8bf5-d38705d64041 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:54.780560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7580b-c9cc-4c7c-8bf5-d38705d64041 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:54.782880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 100/206] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:54 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:54.791301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7580b-c9cc-4c7c-8bf5-d38705d64041 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:54.843979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d760056-c5a9-4aa4-ad06-905c97a2d1aa None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:54.892043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:54.897430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7831bb3-32d6-42bb-94a7-663eb4332ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:54.898998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831bb3-32d6-42bb-94a7-663eb4332ff6 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:54.899424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831bb3-32d6-42bb-94a7-663eb4332ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:54.899841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7831bb3-32d6-42bb-94a7-663eb4332ff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:54.900717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7831bb3-32d6-42bb-94a7-663eb4332ff6 None admin] RBAC: Adding query filter params (role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=5a9037a124a2489992eba863bf5e9b44, user.id=3d536721c74b47c6adbcee5d5db41557) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:54.901358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7831bb3-32d6-42bb-94a7-663eb4332ff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:54.902957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7831bb3-32d6-42bb-94a7-663eb4332ff6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:54.909303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 107/207] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:54 2022] GET /identity/v3/role_assignments?user.id=3d536721c74b47c6adbcee5d5db41557&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=eb7bedf01e0744eeb26681978db4121c => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:54.964100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7580b-c9cc-4c7c-8bf5-d38705d64041 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:55.012727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1a977-64d2-46f7-a8e2-a9b6a169cffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:55.013622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1a977-64d2-46f7-a8e2-a9b6a169cffa None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:55.014047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1a977-64d2-46f7-a8e2-a9b6a169cffa None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:55.014462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1a977-64d2-46f7-a8e2-a9b6a169cffa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:55.015335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1a977-64d2-46f7-a8e2-a9b6a169cffa None admin] RBAC: Adding query filter params (role.id=8041ef90cc68470fa80216023a3091c8, scope.project.id=b6271cacfed747da992ff32e5f4813f9, user.id=ecbecebf1fc5466393a2a83eb2488e94) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:55.015815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1a977-64d2-46f7-a8e2-a9b6a169cffa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:55.017431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1a977-64d2-46f7-a8e2-a9b6a169cffa None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:55.028945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 101/208] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:54 2022] GET /identity/v3/role_assignments?user.id=ecbecebf1fc5466393a2a83eb2488e94&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=8041ef90cc68470fa80216023a3091c8 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:55.880604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea65689-6826-4be4-84f8-bbf44fb67823 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:55.881144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea65689-6826-4be4-84f8-bbf44fb67823 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:55.881254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea65689-6826-4be4-84f8-bbf44fb67823 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:55.882869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 108/209] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:55.888399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75067ce-8bbd-4982-abec-905ae2a29a69 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:55.889278 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75067ce-8bbd-4982-abec-905ae2a29a69 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:55.889772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75067ce-8bbd-4982-abec-905ae2a29a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:55.920671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c75067ce-8bbd-4982-abec-905ae2a29a69 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:55.922491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c75067ce-8bbd-4982-abec-905ae2a29a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:55.995163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 102/210] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:56.038342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f55cc-fbec-4132-b7c5-77646ea266f0 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.038946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f55cc-fbec-4132-b7c5-77646ea266f0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.039439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f55cc-fbec-4132-b7c5-77646ea266f0 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.040970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 109/211] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:56.047085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ab3d5-4f75-43be-9b46-d4c4125ffcf4 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.047578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ab3d5-4f75-43be-9b46-d4c4125ffcf4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.048062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ab3d5-4f75-43be-9b46-d4c4125ffcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.066577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e31ab3d5-4f75-43be-9b46-d4c4125ffcf4 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:56.068044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e31ab3d5-4f75-43be-9b46-d4c4125ffcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:56.112883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:56.139058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 103/212] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:56.277707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f55cc-fbec-4132-b7c5-77646ea266f0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:56.384907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f55cc-fbec-4132-b7c5-77646ea266f0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:56.394396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:56.422095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a1409-50d2-409b-aaa9-ea46a41cd914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:56.423108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a1409-50d2-409b-aaa9-ea46a41cd914 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.423606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a1409-50d2-409b-aaa9-ea46a41cd914 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.424079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a1409-50d2-409b-aaa9-ea46a41cd914 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.430545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ab3d5-4f75-43be-9b46-d4c4125ffcf4 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:56.438957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a1409-50d2-409b-aaa9-ea46a41cd914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-967a1409-50d2-409b-aaa9-ea46a41cd914 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:56.444298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:56.445248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:56.450262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 110/213] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:56.453886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a1409-50d2-409b-aaa9-ea46a41cd914 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:56.557506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ab3d5-4f75-43be-9b46-d4c4125ffcf4 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:56.584823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a1409-50d2-409b-aaa9-ea46a41cd914 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:56.596717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52ac40-4239-4956-ac0d-771d03e1a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:56.597710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52ac40-4239-4956-ac0d-771d03e1a63f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.598213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52ac40-4239-4956-ac0d-771d03e1a63f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.598705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52ac40-4239-4956-ac0d-771d03e1a63f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.600315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52ac40-4239-4956-ac0d-771d03e1a63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:56.602548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52ac40-4239-4956-ac0d-771d03e1a63f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:56.605086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 104/214] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:34:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:56.610719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52ac40-4239-4956-ac0d-771d03e1a63f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:56.627947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b8baf-0874-4e56-880f-80890a6b4a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:56.628670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8baf-0874-4e56-880f-80890a6b4a6b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.628954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8baf-0874-4e56-880f-80890a6b4a6b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.629269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b8baf-0874-4e56-880f-80890a6b4a6b None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.629936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b8baf-0874-4e56-880f-80890a6b4a6b None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:56.630503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b8baf-0874-4e56-880f-80890a6b4a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:56.631888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b8baf-0874-4e56-880f-80890a6b4a6b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:56.640017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 111/215] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:34:56.641491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d090f-5d54-4ba8-a3f8-bbd5075ac585 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.641776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d090f-5d54-4ba8-a3f8-bbd5075ac585 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.642046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d090f-5d54-4ba8-a3f8-bbd5075ac585 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.643249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 112/216] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:56.645483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d090f-5d54-4ba8-a3f8-bbd5075ac585 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:56.668555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:56.681254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:56.729385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52ac40-4239-4956-ac0d-771d03e1a63f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:56.754743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d090f-5d54-4ba8-a3f8-bbd5075ac585 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:56.769456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d138a9-3e7c-40ff-9ce0-d948ea89a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:56.770063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d138a9-3e7c-40ff-9ce0-d948ea89a507 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.770343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d138a9-3e7c-40ff-9ce0-d948ea89a507 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.770615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d138a9-3e7c-40ff-9ce0-d948ea89a507 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.771266 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d138a9-3e7c-40ff-9ce0-d948ea89a507 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:56.772767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d138a9-3e7c-40ff-9ce0-d948ea89a507 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:56.792663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fde932-07b7-465f-8aad-b0a9b87f60be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:56.793462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde932-07b7-465f-8aad-b0a9b87f60be None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.793743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde932-07b7-465f-8aad-b0a9b87f60be None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.794021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde932-07b7-465f-8aad-b0a9b87f60be None admin] PATH_INFO: `/v3/users/9f49ecd8839f41f8be93455b21613c20` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.797627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 105/217] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:34:56 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:34:56.809334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7082529-9dac-4af2-892c-2268be606de0 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.809334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7082529-9dac-4af2-892c-2268be606de0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.809334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7082529-9dac-4af2-892c-2268be606de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:56.817979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fde932-07b7-465f-8aad-b0a9b87f60be None admin] RBAC: Authorizing `identity:get_user(user_id=9f49ecd8839f41f8be93455b21613c20)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:56.819641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fde932-07b7-465f-8aad-b0a9b87f60be None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:56.821706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 113/218] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:56 2022] GET /identity/v3/users/9f49ecd8839f41f8be93455b21613c20 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:56.827826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c7082529-9dac-4af2-892c-2268be606de0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:56.829086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c7082529-9dac-4af2-892c-2268be606de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:56.838133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fde932-07b7-465f-8aad-b0a9b87f60be None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:56.930516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 106/219] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:56.940830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:56.955085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fde932-07b7-465f-8aad-b0a9b87f60be None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:56.994591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca631c-88aa-4177-9d93-f7236090d745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:56.995258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca631c-88aa-4177-9d93-f7236090d745 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:56.995532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca631c-88aa-4177-9d93-f7236090d745 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:56.995848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca631c-88aa-4177-9d93-f7236090d745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:57.065847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca631c-88aa-4177-9d93-f7236090d745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:57.067318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca631c-88aa-4177-9d93-f7236090d745 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:57.161194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:57.173355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 114/220] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:34:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:34:57.178935 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7082529-9dac-4af2-892c-2268be606de0 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:57.215109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca631c-88aa-4177-9d93-f7236090d745 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:57.218320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:57.305927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7082529-9dac-4af2-892c-2268be606de0 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:57.352577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca631c-88aa-4177-9d93-f7236090d745 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:57.371491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed989b88-b06f-467a-a65b-c63bc6087c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:57.372156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989b88-b06f-467a-a65b-c63bc6087c62 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:57.372433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989b88-b06f-467a-a65b-c63bc6087c62 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:57.372701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed989b88-b06f-467a-a65b-c63bc6087c62 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:57.378340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed989b88-b06f-467a-a65b-c63bc6087c62 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:57.379825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed989b88-b06f-467a-a65b-c63bc6087c62 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:57.381839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 107/221] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:57 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:57.387727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed989b88-b06f-467a-a65b-c63bc6087c62 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:57.418970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5aaf6-0052-4b43-8fb6-baed913a4567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:57.419523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5aaf6-0052-4b43-8fb6-baed913a4567 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:57.419798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5aaf6-0052-4b43-8fb6-baed913a4567 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:57.420068 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5aaf6-0052-4b43-8fb6-baed913a4567 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:57.451759 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5aaf6-0052-4b43-8fb6-baed913a4567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-95a5aaf6-0052-4b43-8fb6-baed913a4567 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:57.464368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:57.465320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:34:57.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 115/222] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:34:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:34:57.473839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134ae4f-26c3-48c1-9091-8af5560b4c39 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:57.474126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134ae4f-26c3-48c1-9091-8af5560b4c39 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:57.474404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134ae4f-26c3-48c1-9091-8af5560b4c39 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:57.475954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 116/223] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:57.476539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134ae4f-26c3-48c1-9091-8af5560b4c39 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:57.541895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:57.545947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed989b88-b06f-467a-a65b-c63bc6087c62 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:57.550684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:57.584641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eadf4c-bcec-4d84-8e5e-7a38068a805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:57.585457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eadf4c-bcec-4d84-8e5e-7a38068a805c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:57.585840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eadf4c-bcec-4d84-8e5e-7a38068a805c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:57.586225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eadf4c-bcec-4d84-8e5e-7a38068a805c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:57.587054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eadf4c-bcec-4d84-8e5e-7a38068a805c None admin] RBAC: Adding query filter params (role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=5a9037a124a2489992eba863bf5e9b44, user.id=9f49ecd8839f41f8be93455b21613c20) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:57.587556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eadf4c-bcec-4d84-8e5e-7a38068a805c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:57.589148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eadf4c-bcec-4d84-8e5e-7a38068a805c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:57.594936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 108/224] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:34:57 2022] GET /identity/v3/role_assignments?user.id=9f49ecd8839f41f8be93455b21613c20&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=18bebd2151824cdfa386be482ff8114b => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:57.601225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01aaa-340f-4cab-aff3-4c3c3553e3c8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:57.601225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01aaa-340f-4cab-aff3-4c3c3553e3c8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:57.601225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab01aaa-340f-4cab-aff3-4c3c3553e3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:57.617387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4ab01aaa-340f-4cab-aff3-4c3c3553e3c8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:57.618746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4ab01aaa-340f-4cab-aff3-4c3c3553e3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:57.654425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134ae4f-26c3-48c1-9091-8af5560b4c39 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:57.711135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 109/225] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:57.734454 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:57.735153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:57.735372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:57.735691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:57.736374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:57.736675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:57.738235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:57.759851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 117/226] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:34:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:57.765995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab01aaa-340f-4cab-aff3-4c3c3553e3c8 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:57.875131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:57.888018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab01aaa-340f-4cab-aff3-4c3c3553e3c8 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:57.928656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ac113-be80-416f-a019-ed14d42be387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:57.929378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ac113-be80-416f-a019-ed14d42be387 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:57.929655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ac113-be80-416f-a019-ed14d42be387 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:57.929921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ac113-be80-416f-a019-ed14d42be387 None admin] PATH_INFO: `/v3/users/3d536721c74b47c6adbcee5d5db41557` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:57.933733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ac113-be80-416f-a019-ed14d42be387 None admin] RBAC: Authorizing `identity:get_user(user_id=3d536721c74b47c6adbcee5d5db41557)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:57.935164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ac113-be80-416f-a019-ed14d42be387 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:57.936759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 110/227] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:57 2022] GET /identity/v3/users/3d536721c74b47c6adbcee5d5db41557 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:57.942631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:58.077432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ac113-be80-416f-a019-ed14d42be387 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:58.104079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173a91-141c-4a85-92b1-a7eef12a46a9 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:58.117309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:58.165997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42a5a9-38aa-4360-9513-1222027e1fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:58.166732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a5a9-38aa-4360-9513-1222027e1fe2 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:58.167008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a5a9-38aa-4360-9513-1222027e1fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:58.167273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42a5a9-38aa-4360-9513-1222027e1fe2 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:58.169274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42a5a9-38aa-4360-9513-1222027e1fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:58.170700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42a5a9-38aa-4360-9513-1222027e1fe2 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:58.174931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 118/228] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:57 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:58.180779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42a5a9-38aa-4360-9513-1222027e1fe2 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:58.236186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ac113-be80-416f-a019-ed14d42be387 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:58.303133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810e59a-20f5-4229-930f-2dc5a557cedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:58.303769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e59a-20f5-4229-930f-2dc5a557cedb None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:58.304041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e59a-20f5-4229-930f-2dc5a557cedb None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:58.304307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e59a-20f5-4229-930f-2dc5a557cedb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:58.325942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f810e59a-20f5-4229-930f-2dc5a557cedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-f810e59a-20f5-4229-930f-2dc5a557cedb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:34:58.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:34:58.332523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:34:58.333343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:34:58.335089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 111/229] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:34:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:34:58.344392 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810e59a-20f5-4229-930f-2dc5a557cedb None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:58.365547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42a5a9-38aa-4360-9513-1222027e1fe2 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:58.390008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:58.431316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b722a44-00fd-4a30-a26a-f206350f5a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:58.432043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b722a44-00fd-4a30-a26a-f206350f5a66 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:58.432867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b722a44-00fd-4a30-a26a-f206350f5a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:58.433098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b722a44-00fd-4a30-a26a-f206350f5a66 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44/users/3d536721c74b47c6adbcee5d5db41557/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:58.436943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b722a44-00fd-4a30-a26a-f206350f5a66 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a9037a124a2489992eba863bf5e9b44, user_id=3d536721c74b47c6adbcee5d5db41557, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:58.438480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b722a44-00fd-4a30-a26a-f206350f5a66 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:58.473089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 119/230] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:34:58 2022] PUT /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44/users/3d536721c74b47c6adbcee5d5db41557/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:34:58.491361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810e59a-20f5-4229-930f-2dc5a557cedb None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:58.540772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:58.542331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:58.542815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:58.543283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:58.544184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:34:58.544715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:58.546378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:58.560396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 112/231] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:34:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:58.561878 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b722a44-00fd-4a30-a26a-f206350f5a66 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:58.632320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:58.712329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b722a44-00fd-4a30-a26a-f206350f5a66 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:58.750566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c0437-a4aa-4292-b103-1cdf6cfc9a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:58.751214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0437-a4aa-4292-b103-1cdf6cfc9a9d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:58.751497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0437-a4aa-4292-b103-1cdf6cfc9a9d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:58.751772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c0437-a4aa-4292-b103-1cdf6cfc9a9d None admin] PATH_INFO: `/v3/users/3d536721c74b47c6adbcee5d5db41557` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:58.752938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c0437-a4aa-4292-b103-1cdf6cfc9a9d None admin] RBAC: Authorizing `identity:get_user(user_id=3d536721c74b47c6adbcee5d5db41557)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:58.754511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c0437-a4aa-4292-b103-1cdf6cfc9a9d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:58.756780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 120/232] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:34:58 2022] GET /identity/v3/users/3d536721c74b47c6adbcee5d5db41557 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:58.762265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:58.851329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:58.871004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe727f-17d6-45bf-a595-7746046a1670 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:58.929448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096687f8-2594-4807-b91e-c1eb3ea1aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:58.930474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096687f8-2594-4807-b91e-c1eb3ea1aab0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:58.930927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096687f8-2594-4807-b91e-c1eb3ea1aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:58.931524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096687f8-2594-4807-b91e-c1eb3ea1aab0 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:58.936266 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096687f8-2594-4807-b91e-c1eb3ea1aab0 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:58.938449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096687f8-2594-4807-b91e-c1eb3ea1aab0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:58.940631 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 113/233] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:34:58 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:34:58.945980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c0437-a4aa-4292-b103-1cdf6cfc9a9d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:34:59.085180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c0437-a4aa-4292-b103-1cdf6cfc9a9d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:34:59.094103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:34:59.137812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e4cda-50f3-4ac2-9edc-c33e9dd41b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:34:59.138475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4cda-50f3-4ac2-9edc-c33e9dd41b09 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:59.138752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4cda-50f3-4ac2-9edc-c33e9dd41b09 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:59.139020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e4cda-50f3-4ac2-9edc-c33e9dd41b09 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44/users/3d536721c74b47c6adbcee5d5db41557/roles/eb7bedf01e0744eeb26681978db4121c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:59.141598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4cda-50f3-4ac2-9edc-c33e9dd41b09 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a9037a124a2489992eba863bf5e9b44, user_id=3d536721c74b47c6adbcee5d5db41557, role_id=eb7bedf01e0744eeb26681978db4121c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:34:59.143052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e4cda-50f3-4ac2-9edc-c33e9dd41b09 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:34:59.158426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 121/234] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:34:58 2022] PUT /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44/users/3d536721c74b47c6adbcee5d5db41557/roles/eb7bedf01e0744eeb26681978db4121c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:34:59.644230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb672-017d-46c3-98c2-2cf8a3f92490 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:59.644935 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb672-017d-46c3-98c2-2cf8a3f92490 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:59.645460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb672-017d-46c3-98c2-2cf8a3f92490 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:59.647012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 114/235] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:59.653218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4789154-321a-4d4c-9c19-745d865fc6d5 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:59.653732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4789154-321a-4d4c-9c19-745d865fc6d5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:59.654213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4789154-321a-4d4c-9c19-745d865fc6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:59.668435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069b995-96ad-4784-b47e-6ee920f83b52 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:59.669023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069b995-96ad-4784-b47e-6ee920f83b52 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:59.669521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069b995-96ad-4784-b47e-6ee920f83b52 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:59.672075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 115/236] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:34:59.675572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f4789154-321a-4d4c-9c19-745d865fc6d5 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:59.676725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f4789154-321a-4d4c-9c19-745d865fc6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:59.677123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6403-a88b-4c6e-acde-c7da06428e96 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:34:59.677595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6403-a88b-4c6e-acde-c7da06428e96 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:34:59.678053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a6403-a88b-4c6e-acde-c7da06428e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:34:59.700728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-345a6403-a88b-4c6e-acde-c7da06428e96 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:34:59.702720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-345a6403-a88b-4c6e-acde-c7da06428e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:34:59.788178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 122/237] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:34:59.830648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 116/238] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:00.120200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4789154-321a-4d4c-9c19-745d865fc6d5 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:00.191367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a6403-a88b-4c6e-acde-c7da06428e96 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:00.229582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:00.236746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4789154-321a-4d4c-9c19-745d865fc6d5 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:00.274541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b395057-217a-4a91-b311-4f20d8451956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:00.275518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b395057-217a-4a91-b311-4f20d8451956 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:00.276049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b395057-217a-4a91-b311-4f20d8451956 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:00.276594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b395057-217a-4a91-b311-4f20d8451956 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:00.277922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b395057-217a-4a91-b311-4f20d8451956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:00.279544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b395057-217a-4a91-b311-4f20d8451956 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:00.281630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 123/239] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:00.287165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b395057-217a-4a91-b311-4f20d8451956 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:00.326469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:00.356289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a6403-a88b-4c6e-acde-c7da06428e96 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:00.392697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b395057-217a-4a91-b311-4f20d8451956 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:00.424553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989eb54-b33d-494f-9a1f-379794f969a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:00.425222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989eb54-b33d-494f-9a1f-379794f969a0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:00.425494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989eb54-b33d-494f-9a1f-379794f969a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:00.425773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989eb54-b33d-494f-9a1f-379794f969a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:00.429724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4db21d-49b8-47f9-91e5-fccd765341b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:00.430541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db21d-49b8-47f9-91e5-fccd765341b6 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:00.431042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db21d-49b8-47f9-91e5-fccd765341b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:00.431499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4db21d-49b8-47f9-91e5-fccd765341b6 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:00.432362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4db21d-49b8-47f9-91e5-fccd765341b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:00.434534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4db21d-49b8-47f9-91e5-fccd765341b6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:00.436454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989eb54-b33d-494f-9a1f-379794f969a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-4989eb54-b33d-494f-9a1f-379794f969a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:00.441358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:00.442261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:00.443046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 117/240] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:00.444142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dcdf8-bac9-415a-8be9-c1d262250d5b None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:00.444357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dcdf8-bac9-415a-8be9-c1d262250d5b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:00.444631 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429dcdf8-bac9-415a-8be9-c1d262250d5b None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:00.446411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 118/241] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:00.451751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dcdf8-bac9-415a-8be9-c1d262250d5b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:00.463941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 124/242] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:35:00 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:35:00.478128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d347-d96a-4e06-880a-7c4ff8fb0ec6 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:00.479368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d347-d96a-4e06-880a-7c4ff8fb0ec6 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:00.479865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d347-d96a-4e06-880a-7c4ff8fb0ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:00.509400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7fc4d347-d96a-4e06-880a-7c4ff8fb0ec6 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:00.516291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7fc4d347-d96a-4e06-880a-7c4ff8fb0ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:00.554382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:00.563597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429dcdf8-bac9-415a-8be9-c1d262250d5b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:00.612286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 125/243] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:00.636209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:00.636870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:00.637187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:00.637462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:00.638143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:00.638448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:00.639849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:00.649435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 119/244] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:00.655423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4d347-d96a-4e06-880a-7c4ff8fb0ec6 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:00.815803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4d347-d96a-4e06-880a-7c4ff8fb0ec6 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:00.856862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a72bb-e245-4c95-9ec0-b955920a6a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:00.857545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a72bb-e245-4c95-9ec0-b955920a6a89 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:00.857821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a72bb-e245-4c95-9ec0-b955920a6a89 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:00.858089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a72bb-e245-4c95-9ec0-b955920a6a89 None admin] PATH_INFO: `/v3/users/e426f095aff24f048868e266239a8ada` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:00.874844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a72bb-e245-4c95-9ec0-b955920a6a89 None admin] RBAC: Authorizing `identity:get_user(user_id=e426f095aff24f048868e266239a8ada)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:00.876346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a72bb-e245-4c95-9ec0-b955920a6a89 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:00.878371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 126/245] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:00 2022] GET /identity/v3/users/e426f095aff24f048868e266239a8ada => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:00.883949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:00.887475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:00.980227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a72bb-e245-4c95-9ec0-b955920a6a89 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:00.995188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:01.013777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d883-6189-4d10-a4c4-4586990a36e0 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:01.062858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75de83-03a0-4de0-9289-01a0f742e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:01.063483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75de83-03a0-4de0-9289-01a0f742e939 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:01.063761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75de83-03a0-4de0-9289-01a0f742e939 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:01.064033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75de83-03a0-4de0-9289-01a0f742e939 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:01.114998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a72bb-e245-4c95-9ec0-b955920a6a89 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:01.154908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75de83-03a0-4de0-9289-01a0f742e939 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:01.156647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75de83-03a0-4de0-9289-01a0f742e939 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:01.159330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:01.165220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddcf77-afda-4170-88b7-087a57cfb0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:01.165933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddcf77-afda-4170-88b7-087a57cfb0ae None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:01.166914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddcf77-afda-4170-88b7-087a57cfb0ae None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:01.167328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cddcf77-afda-4170-88b7-087a57cfb0ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:01.181192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cddcf77-afda-4170-88b7-087a57cfb0ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-4cddcf77-afda-4170-88b7-087a57cfb0ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:01.187972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:01.189053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:01.190091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:01.192218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 127/246] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:01.198976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddcf77-afda-4170-88b7-087a57cfb0ae None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:01.281935 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 120/247] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:01.283784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7e0ed-4890-4649-a47c-f12ffcb332dc None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:01.284282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7e0ed-4890-4649-a47c-f12ffcb332dc None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:01.284753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7e0ed-4890-4649-a47c-f12ffcb332dc None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:01.286255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 121/248] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:01.288934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:01.291933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7e0ed-4890-4649-a47c-f12ffcb332dc None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:01.337925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cddcf77-afda-4170-88b7-087a57cfb0ae None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:01.380013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e746d1-76bf-4fa8-817e-fb730a213f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:01.381512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e746d1-76bf-4fa8-817e-fb730a213f45 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:01.381994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e746d1-76bf-4fa8-817e-fb730a213f45 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:01.382639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e746d1-76bf-4fa8-817e-fb730a213f45 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:01.384294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e746d1-76bf-4fa8-817e-fb730a213f45 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:01.384806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e746d1-76bf-4fa8-817e-fb730a213f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:01.386926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e746d1-76bf-4fa8-817e-fb730a213f45 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:01.400412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 128/249] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:01.405039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20f325-ca60-4f4d-b054-5f19bfb56e9a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:01.405039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20f325-ca60-4f4d-b054-5f19bfb56e9a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:01.405039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20f325-ca60-4f4d-b054-5f19bfb56e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:01.424021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-3e20f325-ca60-4f4d-b054-5f19bfb56e9a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:01.424366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7e0ed-4890-4649-a47c-f12ffcb332dc None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:01.425253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3e20f325-ca60-4f4d-b054-5f19bfb56e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:01.426592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:01.466013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c44c9e-9a74-4de3-9fd9-05507ced5c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:01.467023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c44c9e-9a74-4de3-9fd9-05507ced5c41 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:01.467499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c44c9e-9a74-4de3-9fd9-05507ced5c41 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:01.467991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c44c9e-9a74-4de3-9fd9-05507ced5c41 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:01.469336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c44c9e-9a74-4de3-9fd9-05507ced5c41 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:01.470977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c44c9e-9a74-4de3-9fd9-05507ced5c41 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:01.472895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 122/250] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:01 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:01.473874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c44c9e-9a74-4de3-9fd9-05507ced5c41 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:01.522632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 129/251] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:01.523908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20f325-ca60-4f4d-b054-5f19bfb56e9a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:01.661981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:01.691027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c44c9e-9a74-4de3-9fd9-05507ced5c41 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:01.732640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20f325-ca60-4f4d-b054-5f19bfb56e9a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:01.782667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9935b51-5b00-4801-9e48-98b59b602a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:01.783690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935b51-5b00-4801-9e48-98b59b602a1c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:01.784167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935b51-5b00-4801-9e48-98b59b602a1c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:01.784632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9935b51-5b00-4801-9e48-98b59b602a1c None admin] PATH_INFO: `/v3/users/9f49ecd8839f41f8be93455b21613c20` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:01.786005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9935b51-5b00-4801-9e48-98b59b602a1c None admin] RBAC: Authorizing `identity:get_user(user_id=9f49ecd8839f41f8be93455b21613c20)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:01.787789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9935b51-5b00-4801-9e48-98b59b602a1c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:01.789623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 123/252] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:01 2022] GET /identity/v3/users/9f49ecd8839f41f8be93455b21613c20 => generated 297 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:01.798352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9935b51-5b00-4801-9e48-98b59b602a1c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:01.800135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:01.800760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:01.801040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:01.801360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:01.802439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] RBAC: Adding query filter params (role.id=8041ef90cc68470fa80216023a3091c8, scope.project.id=5a9037a124a2489992eba863bf5e9b44, user.id=e426f095aff24f048868e266239a8ada) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:01.802753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:01.805337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:01.819348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 130/253] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:01 2022] GET /identity/v3/role_assignments?user.id=e426f095aff24f048868e266239a8ada&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=8041ef90cc68470fa80216023a3091c8 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:01.876106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:01.949603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:01.962933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9935b51-5b00-4801-9e48-98b59b602a1c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:02.008716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f9640-8ae2-4e2a-8f2b-d965af6f72a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:02.009403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f9640-8ae2-4e2a-8f2b-d965af6f72a7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:02.009684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f9640-8ae2-4e2a-8f2b-d965af6f72a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:02.010479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f9640-8ae2-4e2a-8f2b-d965af6f72a7 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:02.011635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f9640-8ae2-4e2a-8f2b-d965af6f72a7 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:02.013110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f9640-8ae2-4e2a-8f2b-d965af6f72a7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:02.014872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 124/254] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:01 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:02.020793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f9640-8ae2-4e2a-8f2b-d965af6f72a7 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:02.085413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59bc75-15ca-4f32-820e-802c90b816ca None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:02.121462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:02.128924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff3038-fab5-42ec-b81b-ccffbeb74e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:02.129609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3038-fab5-42ec-b81b-ccffbeb74e03 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:02.129885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3038-fab5-42ec-b81b-ccffbeb74e03 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:02.131608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff3038-fab5-42ec-b81b-ccffbeb74e03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:02.154872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff3038-fab5-42ec-b81b-ccffbeb74e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-6eff3038-fab5-42ec-b81b-ccffbeb74e03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:02.161288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:02.162232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:02.163000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:02.166091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 131/255] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:35:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:02.176033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28233f-d581-45f0-b16e-55d3b1680585 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:02.176330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28233f-d581-45f0-b16e-55d3b1680585 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:02.176565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28233f-d581-45f0-b16e-55d3b1680585 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:02.177998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 132/256] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:02 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:02.178733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28233f-d581-45f0-b16e-55d3b1680585 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:02.207256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f9640-8ae2-4e2a-8f2b-d965af6f72a7 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:02.220997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:02.260531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9365-fb78-43ad-85ca-9078e2ae1f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:02.261663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9365-fb78-43ad-85ca-9078e2ae1f12 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:02.262167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9365-fb78-43ad-85ca-9078e2ae1f12 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:02.262672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9365-fb78-43ad-85ca-9078e2ae1f12 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44/users/9f49ecd8839f41f8be93455b21613c20/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:02.264901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c9365-fb78-43ad-85ca-9078e2ae1f12 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a9037a124a2489992eba863bf5e9b44, user_id=9f49ecd8839f41f8be93455b21613c20, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:02.266581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c9365-fb78-43ad-85ca-9078e2ae1f12 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:02.296321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 125/257] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:35:02 2022] PUT /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44/users/9f49ecd8839f41f8be93455b21613c20/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:02.302642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254fb74-0d22-46bc-80f8-63358294b196 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:02.303244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254fb74-0d22-46bc-80f8-63358294b196 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:02.303720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254fb74-0d22-46bc-80f8-63358294b196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:02.334520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f254fb74-0d22-46bc-80f8-63358294b196 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:02.336131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f254fb74-0d22-46bc-80f8-63358294b196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:02.348869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28233f-d581-45f0-b16e-55d3b1680585 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:02.434207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:02.434871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:02.435154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:02.435420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:02.436081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:02.436435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:02.440190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:02.461357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 133/258] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:02.466307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:02.484407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 126/259] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:02.529023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:02.596882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5ac5c-aedf-43e1-8d53-aa636350b0d7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:02.634902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a76956-377f-42dd-b118-4ae75cc516c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:02.635938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a76956-377f-42dd-b118-4ae75cc516c7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:02.636446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a76956-377f-42dd-b118-4ae75cc516c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:02.636918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a76956-377f-42dd-b118-4ae75cc516c7 None admin] PATH_INFO: `/v3/users/3d536721c74b47c6adbcee5d5db41557` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:02.638298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a76956-377f-42dd-b118-4ae75cc516c7 None admin] RBAC: Authorizing `identity:get_user(user_id=3d536721c74b47c6adbcee5d5db41557)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:02.639959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a76956-377f-42dd-b118-4ae75cc516c7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:02.642170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 134/260] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:02 2022] GET /identity/v3/users/3d536721c74b47c6adbcee5d5db41557 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:02.647382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254fb74-0d22-46bc-80f8-63358294b196 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:02.778203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:02.784734 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254fb74-0d22-46bc-80f8-63358294b196 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:02.828544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552de058-7686-4a4c-be8d-04fd033522d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:02.829180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552de058-7686-4a4c-be8d-04fd033522d9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:02.829457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552de058-7686-4a4c-be8d-04fd033522d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:02.829723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552de058-7686-4a4c-be8d-04fd033522d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:02.899930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552de058-7686-4a4c-be8d-04fd033522d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:02.901997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552de058-7686-4a4c-be8d-04fd033522d9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:02.979188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a76956-377f-42dd-b118-4ae75cc516c7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:02.997544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:03.006275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 127/261] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:03.014098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552de058-7686-4a4c-be8d-04fd033522d9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:03.124111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552de058-7686-4a4c-be8d-04fd033522d9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:03.183553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f609b-7a0e-4069-bb7e-285f76575c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:03.184575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f609b-7a0e-4069-bb7e-285f76575c82 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:03.185074 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f609b-7a0e-4069-bb7e-285f76575c82 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:03.185555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f609b-7a0e-4069-bb7e-285f76575c82 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:03.186974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f609b-7a0e-4069-bb7e-285f76575c82 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:03.193169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f609b-7a0e-4069-bb7e-285f76575c82 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:03.195587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 128/262] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:03 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:03.204198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f609b-7a0e-4069-bb7e-285f76575c82 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:03.239213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a76956-377f-42dd-b118-4ae75cc516c7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:03.269408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:03.305301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:03.317773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517ee99-9068-4e99-b9cd-3942784fe55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:03.318427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517ee99-9068-4e99-b9cd-3942784fe55c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:03.318704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517ee99-9068-4e99-b9cd-3942784fe55c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:03.318976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517ee99-9068-4e99-b9cd-3942784fe55c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:03.346296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517ee99-9068-4e99-b9cd-3942784fe55c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:03.347884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f609b-7a0e-4069-bb7e-285f76575c82 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-5517ee99-9068-4e99-b9cd-3942784fe55c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:03.363521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:03.364480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:03.365363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 135/263] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:35:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:03.370599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517ee99-9068-4e99-b9cd-3942784fe55c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:03.391534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:03.392131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:03.392409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:03.392679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:03.393443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=5a9037a124a2489992eba863bf5e9b44, user.id=3d536721c74b47c6adbcee5d5db41557) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:03.393746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:03.395178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:03.409665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 129/264] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:03 2022] GET /identity/v3/role_assignments?user.id=3d536721c74b47c6adbcee5d5db41557&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=b131b61299154544a627315649090a49 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:03.569601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517ee99-9068-4e99-b9cd-3942784fe55c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:03.619720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6805a-eeba-4c78-9590-295f6bbd7bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:03.620344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6805a-eeba-4c78-9590-295f6bbd7bed None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:03.620624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6805a-eeba-4c78-9590-295f6bbd7bed None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:03.620893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6805a-eeba-4c78-9590-295f6bbd7bed None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:03.621598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6805a-eeba-4c78-9590-295f6bbd7bed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:03.621900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6805a-eeba-4c78-9590-295f6bbd7bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:03.623304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6805a-eeba-4c78-9590-295f6bbd7bed None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:03.632997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 136/265] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:03.639703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:03.665843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6ad0d-257b-4795-b84f-83af6eff292f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:03.666120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6ad0d-257b-4795-b84f-83af6eff292f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:03.666868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6ad0d-257b-4795-b84f-83af6eff292f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:03.668493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 137/266] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:03.673511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f4c0-e7b9-4f2a-a59c-cc98204d28a3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:03.673794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f4c0-e7b9-4f2a-a59c-cc98204d28a3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:03.674064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24f4c0-e7b9-4f2a-a59c-cc98204d28a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:03.707131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-3e24f4c0-e7b9-4f2a-a59c-cc98204d28a3 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:03.708382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3e24f4c0-e7b9-4f2a-a59c-cc98204d28a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:03.782546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:03.864074 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c8b8-8607-4563-8f0d-62bf362d8c58 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:03.885246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 138/267] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:03.903165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:03.910860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f4f05-6aa1-4de6-a7a8-5fa3c5ff3b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:03.911500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f4f05-6aa1-4de6-a7a8-5fa3c5ff3b6e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:03.911795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f4f05-6aa1-4de6-a7a8-5fa3c5ff3b6e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:03.912048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f4f05-6aa1-4de6-a7a8-5fa3c5ff3b6e None admin] PATH_INFO: `/v3/users/3d536721c74b47c6adbcee5d5db41557` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:03.913209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f4f05-6aa1-4de6-a7a8-5fa3c5ff3b6e None admin] RBAC: Authorizing `identity:get_user(user_id=3d536721c74b47c6adbcee5d5db41557)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:03.914661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f4f05-6aa1-4de6-a7a8-5fa3c5ff3b6e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:03.916762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 130/268] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:03 2022] GET /identity/v3/users/3d536721c74b47c6adbcee5d5db41557 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:03.927062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24f4c0-e7b9-4f2a-a59c-cc98204d28a3 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:04.071846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24f4c0-e7b9-4f2a-a59c-cc98204d28a3 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:04.123105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a09c7-0d05-488c-9ee3-6db7009ecd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:04.123718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a09c7-0d05-488c-9ee3-6db7009ecd46 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.123994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a09c7-0d05-488c-9ee3-6db7009ecd46 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.124264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a09c7-0d05-488c-9ee3-6db7009ecd46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.169484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:04.211034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a09c7-0d05-488c-9ee3-6db7009ecd46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:04.212507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a09c7-0d05-488c-9ee3-6db7009ecd46 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:04.335871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 139/269] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:04.341917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f4f05-6aa1-4de6-a7a8-5fa3c5ff3b6e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:04.378012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a09c7-0d05-488c-9ee3-6db7009ecd46 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:04.482030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:04.501292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f4f05-6aa1-4de6-a7a8-5fa3c5ff3b6e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:04.519147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a09c7-0d05-488c-9ee3-6db7009ecd46 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:04.536113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e838d-55b2-4386-be1d-81170f4fa08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:04.536784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e838d-55b2-4386-be1d-81170f4fa08c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.537080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e838d-55b2-4386-be1d-81170f4fa08c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.537403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e838d-55b2-4386-be1d-81170f4fa08c None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.538522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e838d-55b2-4386-be1d-81170f4fa08c None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:04.539967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e838d-55b2-4386-be1d-81170f4fa08c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:04.542436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 131/270] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:04 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:04.543550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6b89f-420b-4a44-8103-a46f8051189a None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.543918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6b89f-420b-4a44-8103-a46f8051189a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.544191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6b89f-420b-4a44-8103-a46f8051189a None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.545647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 132/271] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:04.548212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6b89f-420b-4a44-8103-a46f8051189a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:04.564832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910451f-f134-4b32-bf2c-cda7f46bc656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:04.565768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910451f-f134-4b32-bf2c-cda7f46bc656 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.566050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910451f-f134-4b32-bf2c-cda7f46bc656 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.566322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910451f-f134-4b32-bf2c-cda7f46bc656 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.580964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910451f-f134-4b32-bf2c-cda7f46bc656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-1910451f-f134-4b32-bf2c-cda7f46bc656 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:04.587040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:04.587918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:04.589244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 140/272] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:04.590190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d4d6a-d8aa-4b80-940a-952df8e198f2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.593082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d4d6a-d8aa-4b80-940a-952df8e198f2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.593082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d4d6a-d8aa-4b80-940a-952df8e198f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.613011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8d2d4d6a-d8aa-4b80-940a-952df8e198f2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:04.614934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8d2d4d6a-d8aa-4b80-940a-952df8e198f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:04.620708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:04.662294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6b89f-420b-4a44-8103-a46f8051189a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:04.700332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e532ae6-825d-4c28-b0ca-8e9f86f552f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:04.701155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532ae6-825d-4c28-b0ca-8e9f86f552f9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.701531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532ae6-825d-4c28-b0ca-8e9f86f552f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.701947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e532ae6-825d-4c28-b0ca-8e9f86f552f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.702765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e532ae6-825d-4c28-b0ca-8e9f86f552f9 None admin] RBAC: Adding query filter params (role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=5a9037a124a2489992eba863bf5e9b44, user.id=3d536721c74b47c6adbcee5d5db41557) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:04.703168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e532ae6-825d-4c28-b0ca-8e9f86f552f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:04.704740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e532ae6-825d-4c28-b0ca-8e9f86f552f9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:04.708705 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 141/273] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:04.713423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d4d6a-d8aa-4b80-940a-952df8e198f2 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:04.718802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 133/274] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:04 2022] GET /identity/v3/role_assignments?user.id=3d536721c74b47c6adbcee5d5db41557&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=eb7bedf01e0744eeb26681978db4121c => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:04.777697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:04.844171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d4d6a-d8aa-4b80-940a-952df8e198f2 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:04.902635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:04.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.903550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.903823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.904509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:04.904809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:04.909093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:04.909303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9bb3e-0aaf-494f-86c5-ec4503ce0044 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.909303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9bb3e-0aaf-494f-86c5-ec4503ce0044 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.909303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9bb3e-0aaf-494f-86c5-ec4503ce0044 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.910541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 134/275] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:35:04.919489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90145c-0342-49db-8cfc-cfb191a82c2b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:04.922016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 142/276] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:04.922569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90145c-0342-49db-8cfc-cfb191a82c2b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:04.923087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90145c-0342-49db-8cfc-cfb191a82c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:04.928027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:04.943101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1a90145c-0342-49db-8cfc-cfb191a82c2b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:04.944786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1a90145c-0342-49db-8cfc-cfb191a82c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:05.066409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 135/277] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:05.071616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90145c-0342-49db-8cfc-cfb191a82c2b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:05.113347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39d85d-9911-403b-9c12-91eba1f9bc83 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:05.135810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:05.153162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:05.164651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fa433-e27b-48fa-9c05-3a1d9bf78a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:05.165319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fa433-e27b-48fa-9c05-3a1d9bf78a78 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.165591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fa433-e27b-48fa-9c05-3a1d9bf78a78 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.165857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fa433-e27b-48fa-9c05-3a1d9bf78a78 None admin] PATH_INFO: `/v3/users/3275a7332f044e26a4791a02f1810a57` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.181819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fa433-e27b-48fa-9c05-3a1d9bf78a78 None admin] RBAC: Authorizing `identity:get_user(user_id=3275a7332f044e26a4791a02f1810a57)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:05.183968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3fa433-e27b-48fa-9c05-3a1d9bf78a78 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:05.186661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 143/278] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:04 2022] GET /identity/v3/users/3275a7332f044e26a4791a02f1810a57 => generated 293 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:05.193160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fa433-e27b-48fa-9c05-3a1d9bf78a78 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:05.218485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90145c-0342-49db-8cfc-cfb191a82c2b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:05.268604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c10b5-ddf9-40f1-83f2-fcefb1cce2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:05.269291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c10b5-ddf9-40f1-83f2-fcefb1cce2d0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.269567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c10b5-ddf9-40f1-83f2-fcefb1cce2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.269833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c10b5-ddf9-40f1-83f2-fcefb1cce2d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.270942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c10b5-ddf9-40f1-83f2-fcefb1cce2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:05.273247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c10b5-ddf9-40f1-83f2-fcefb1cce2d0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:05.275770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 136/279] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:05.281987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c10b5-ddf9-40f1-83f2-fcefb1cce2d0 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:05.333039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fa433-e27b-48fa-9c05-3a1d9bf78a78 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:05.370122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a643856-3bfb-49f2-a4c9-22a2872561ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:05.371485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643856-3bfb-49f2-a4c9-22a2872561ec None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.371963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643856-3bfb-49f2-a4c9-22a2872561ec None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.372438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643856-3bfb-49f2-a4c9-22a2872561ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.373394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a643856-3bfb-49f2-a4c9-22a2872561ec None admin] RBAC: Adding query filter params (role.id=18bebd2151824cdfa386be482ff8114b, user.id=3275a7332f044e26a4791a02f1810a57, scope.system=all) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:05.374054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a643856-3bfb-49f2-a4c9-22a2872561ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:05.375598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a643856-3bfb-49f2-a4c9-22a2872561ec None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:05.383435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 144/280] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 9 01:35:05 2022] GET /identity/v3/role_assignments?user.id=3275a7332f044e26a4791a02f1810a57&scope.system=all&role.id=18bebd2151824cdfa386be482ff8114b => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:05.383435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aae43-4433-4ef6-b74b-561cd5441bc4 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.383435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aae43-4433-4ef6-b74b-561cd5441bc4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.383435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aae43-4433-4ef6-b74b-561cd5441bc4 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.384076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 145/281] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:05.384604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aae43-4433-4ef6-b74b-561cd5441bc4 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:05.409040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c10b5-ddf9-40f1-83f2-fcefb1cce2d0 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:05.438483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:05.443881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:05.454029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb991d0-0f93-4cca-8782-2deaca01654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:05.454657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb991d0-0f93-4cca-8782-2deaca01654f None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.454941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb991d0-0f93-4cca-8782-2deaca01654f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.455206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb991d0-0f93-4cca-8782-2deaca01654f None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.456111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb991d0-0f93-4cca-8782-2deaca01654f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:05.457633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb991d0-0f93-4cca-8782-2deaca01654f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:05.483434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 137/282] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:35:05 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:35:05.485111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517673f-2b1a-4b1d-85f2-d16c71ffdeb6 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.485591 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517673f-2b1a-4b1d-85f2-d16c71ffdeb6 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.486071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517673f-2b1a-4b1d-85f2-d16c71ffdeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.516453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aae43-4433-4ef6-b74b-561cd5441bc4 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:05.521852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a517673f-2b1a-4b1d-85f2-d16c71ffdeb6 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:05.523171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a517673f-2b1a-4b1d-85f2-d16c71ffdeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:05.567348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2eea4a-852b-45d1-8e90-5f295b4c47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:05.568443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eea4a-852b-45d1-8e90-5f295b4c47ff None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.568869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eea4a-852b-45d1-8e90-5f295b4c47ff None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.569443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eea4a-852b-45d1-8e90-5f295b4c47ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.584384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2eea4a-852b-45d1-8e90-5f295b4c47ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-ac2eea4a-852b-45d1-8e90-5f295b4c47ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:05.589022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:05.589951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:05.590779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 146/283] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:05.596375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2eea4a-852b-45d1-8e90-5f295b4c47ff None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:05.616073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 138/284] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:05.703015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:05.706285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2eea4a-852b-45d1-8e90-5f295b4c47ff None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:05.744311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:05.744928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.745254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.745535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.746203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:05.746504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:05.747966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:05.758246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 147/285] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:05.763974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517673f-2b1a-4b1d-85f2-d16c71ffdeb6 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:05.877896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517673f-2b1a-4b1d-85f2-d16c71ffdeb6 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:05.895482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:05.903088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:05.926388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e61fb-a6e7-40c7-881b-3e797c1eb609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:05.927409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e61fb-a6e7-40c7-881b-3e797c1eb609 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:05.927907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e61fb-a6e7-40c7-881b-3e797c1eb609 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:05.928434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e61fb-a6e7-40c7-881b-3e797c1eb609 None admin] PATH_INFO: `/v3/users/e426f095aff24f048868e266239a8ada` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:05.929843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e61fb-a6e7-40c7-881b-3e797c1eb609 None admin] RBAC: Authorizing `identity:get_user(user_id=e426f095aff24f048868e266239a8ada)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:05.931275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e61fb-a6e7-40c7-881b-3e797c1eb609 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:05.933475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 139/286] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:05 2022] GET /identity/v3/users/e426f095aff24f048868e266239a8ada => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:05.938876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e61fb-a6e7-40c7-881b-3e797c1eb609 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:06.032062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03224e-ed8a-456f-82cc-7a3de56b14a1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:06.053021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e61fb-a6e7-40c7-881b-3e797c1eb609 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:06.070263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d0035-855e-4976-bcb2-f7905365e30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:06.071244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d0035-855e-4976-bcb2-f7905365e30e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:06.071746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d0035-855e-4976-bcb2-f7905365e30e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:06.072218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d0035-855e-4976-bcb2-f7905365e30e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:06.083263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d0035-855e-4976-bcb2-f7905365e30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-142d0035-855e-4976-bcb2-f7905365e30e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:06.089930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:06.090799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:06.091556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:06.093694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 148/287] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:06.094295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf84d-1f6a-4a72-938c-5f90bb311eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:06.095004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf84d-1f6a-4a72-938c-5f90bb311eda None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:06.095349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf84d-1f6a-4a72-938c-5f90bb311eda None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:06.095699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf84d-1f6a-4a72-938c-5f90bb311eda None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:06.101786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bf84d-1f6a-4a72-938c-5f90bb311eda None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:06.102833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d0035-855e-4976-bcb2-f7905365e30e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:06.103174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bf84d-1f6a-4a72-938c-5f90bb311eda None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:06.105251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 140/288] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:05 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:06.114337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf84d-1f6a-4a72-938c-5f90bb311eda None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:06.151642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:06.221517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:06.295198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf84d-1f6a-4a72-938c-5f90bb311eda None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:06.319596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d0035-855e-4976-bcb2-f7905365e30e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:06.333666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669c20a-47bc-4e64-b0f5-cb6f698d25ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:06.341124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c20a-47bc-4e64-b0f5-cb6f698d25ce None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:06.341124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c20a-47bc-4e64-b0f5-cb6f698d25ce None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:06.341124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669c20a-47bc-4e64-b0f5-cb6f698d25ce None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44/users/e426f095aff24f048868e266239a8ada/roles/8041ef90cc68470fa80216023a3091c8` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:06.346516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669c20a-47bc-4e64-b0f5-cb6f698d25ce None admin] RBAC: Authorizing `identity:create_grant(project_id=5a9037a124a2489992eba863bf5e9b44, user_id=e426f095aff24f048868e266239a8ada, role_id=8041ef90cc68470fa80216023a3091c8)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:06.348253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669c20a-47bc-4e64-b0f5-cb6f698d25ce None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:06.371068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 141/289] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:35:06 2022] PUT /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44/users/e426f095aff24f048868e266239a8ada/roles/8041ef90cc68470fa80216023a3091c8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:35:06.385920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:06.387247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:06.387670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:06.388084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:06.388921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:06.389652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:06.391356 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:06.401613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 149/290] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:06.408122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669c20a-47bc-4e64-b0f5-cb6f698d25ce None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:06.542317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669c20a-47bc-4e64-b0f5-cb6f698d25ce None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:06.558633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:06.590486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07d417-2d88-41a2-8e4a-96fb57cef757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:06.591143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07d417-2d88-41a2-8e4a-96fb57cef757 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:06.591420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07d417-2d88-41a2-8e4a-96fb57cef757 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:06.591689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07d417-2d88-41a2-8e4a-96fb57cef757 None admin] PATH_INFO: `/v3/users/9f49ecd8839f41f8be93455b21613c20` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:06.592984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07d417-2d88-41a2-8e4a-96fb57cef757 None admin] RBAC: Authorizing `identity:get_user(user_id=9f49ecd8839f41f8be93455b21613c20)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:06.594606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07d417-2d88-41a2-8e4a-96fb57cef757 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:06.596572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 142/291] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:06 2022] GET /identity/v3/users/9f49ecd8839f41f8be93455b21613c20 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:06.602186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:06.698645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:06.715168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33df059-b27d-47ae-9647-72efc1b8f9d0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:06.751591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e563a-f334-4800-a3ef-53034bae196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:06.752203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e563a-f334-4800-a3ef-53034bae196a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:06.752480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e563a-f334-4800-a3ef-53034bae196a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:06.752752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e563a-f334-4800-a3ef-53034bae196a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:06.830442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e563a-f334-4800-a3ef-53034bae196a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:06.831895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e563a-f334-4800-a3ef-53034bae196a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:06.933427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:06.938662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 150/292] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:06.944362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07d417-2d88-41a2-8e4a-96fb57cef757 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:07.069888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07d417-2d88-41a2-8e4a-96fb57cef757 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:07.112445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdf47d-1519-445a-8995-5a8c3715bee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:07.113093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdf47d-1519-445a-8995-5a8c3715bee5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:07.113382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdf47d-1519-445a-8995-5a8c3715bee5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:07.113655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdf47d-1519-445a-8995-5a8c3715bee5 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:07.114786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdf47d-1519-445a-8995-5a8c3715bee5 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:07.116218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdf47d-1519-445a-8995-5a8c3715bee5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:07.121148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 143/293] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:06 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:07.126198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e563a-f334-4800-a3ef-53034bae196a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:07.271606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e563a-f334-4800-a3ef-53034bae196a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:07.309723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304624e-f5f1-4c9b-a556-537e8b4061ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:07.311032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304624e-f5f1-4c9b-a556-537e8b4061ab None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:07.311702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304624e-f5f1-4c9b-a556-537e8b4061ab None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:07.312365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304624e-f5f1-4c9b-a556-537e8b4061ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:07.314129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304624e-f5f1-4c9b-a556-537e8b4061ab None admin] RBAC: Adding query filter params (role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=5a9037a124a2489992eba863bf5e9b44, user.id=9f49ecd8839f41f8be93455b21613c20) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:07.314994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304624e-f5f1-4c9b-a556-537e8b4061ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:07.317861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304624e-f5f1-4c9b-a556-537e8b4061ab None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:07.327098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 151/294] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:07 2022] GET /identity/v3/role_assignments?user.id=9f49ecd8839f41f8be93455b21613c20&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=18bebd2151824cdfa386be482ff8114b => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:07.924305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67dfc5-3ef0-4d99-9ddd-cf46ce9afe9d None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:07.924814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67dfc5-3ef0-4d99-9ddd-cf46ce9afe9d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:07.924888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67dfc5-3ef0-4d99-9ddd-cf46ce9afe9d None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:07.926362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 144/295] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:07.931307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cced5-bdaa-48a6-ab11-9ae9681c91f0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:07.931765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cced5-bdaa-48a6-ab11-9ae9681c91f0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:07.931964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cced5-bdaa-48a6-ab11-9ae9681c91f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:07.955627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4f5cced5-bdaa-48a6-ab11-9ae9681c91f0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:07.956832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4f5cced5-bdaa-48a6-ab11-9ae9681c91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:08.010377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:08.037413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 152/296] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:08.320141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67dfc5-3ef0-4d99-9ddd-cf46ce9afe9d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:08.343030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd420ce-1321-4437-93bf-413974ac4954 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.343030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd420ce-1321-4437-93bf-413974ac4954 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.343030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd420ce-1321-4437-93bf-413974ac4954 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.343665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 153/297] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:08.349658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782feb1-e303-4c33-a42f-441011cdb8f1 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.350186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782feb1-e303-4c33-a42f-441011cdb8f1 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.350765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782feb1-e303-4c33-a42f-441011cdb8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.372670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6782feb1-e303-4c33-a42f-441011cdb8f1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:08.373623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:08.374461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6782feb1-e303-4c33-a42f-441011cdb8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:08.434728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67dfc5-3ef0-4d99-9ddd-cf46ce9afe9d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:08.457265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 154/298] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:08.470325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ed93a-ab5c-4496-87de-6665a980977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:08.470963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed93a-ab5c-4496-87de-6665a980977b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.471243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed93a-ab5c-4496-87de-6665a980977b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.471516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ed93a-ab5c-4496-87de-6665a980977b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.482233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ed93a-ab5c-4496-87de-6665a980977b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-cb2ed93a-ab5c-4496-87de-6665a980977b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:08.487163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:08.488016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:08.489228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 145/299] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:08.494916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782feb1-e303-4c33-a42f-441011cdb8f1 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:08.512900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d2f83-19b0-445f-89b3-8673a30e9c00 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.513491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d2f83-19b0-445f-89b3-8673a30e9c00 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.513939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d2f83-19b0-445f-89b3-8673a30e9c00 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.515695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 146/300] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:35:08.521026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65d63b-8f02-4c97-a180-bfa58a194aed None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.521604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65d63b-8f02-4c97-a180-bfa58a194aed None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.522056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65d63b-8f02-4c97-a180-bfa58a194aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.541785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-bf65d63b-8f02-4c97-a180-bfa58a194aed None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:08.543508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-bf65d63b-8f02-4c97-a180-bfa58a194aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:08.606031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782feb1-e303-4c33-a42f-441011cdb8f1 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:08.615889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:08.618190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 147/301] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:08.652742 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:08.653383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.653663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.653941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.654592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:08.654898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:08.656256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:08.665785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 155/302] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:08.671463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65d63b-8f02-4c97-a180-bfa58a194aed None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:08.710494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:08.763125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:08.779895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65d63b-8f02-4c97-a180-bfa58a194aed None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:08.813500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1c21e-f708-4173-b225-138a3dbc2260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:08.814127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c21e-f708-4173-b225-138a3dbc2260 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.814406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c21e-f708-4173-b225-138a3dbc2260 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.814749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1c21e-f708-4173-b225-138a3dbc2260 None admin] PATH_INFO: `/v3/users/3275a7332f044e26a4791a02f1810a57` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.815868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1c21e-f708-4173-b225-138a3dbc2260 None admin] RBAC: Authorizing `identity:get_user(user_id=3275a7332f044e26a4791a02f1810a57)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:08.817322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1c21e-f708-4173-b225-138a3dbc2260 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:08.818919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 148/303] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:08 2022] GET /identity/v3/users/3275a7332f044e26a4791a02f1810a57 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:08.824844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1c21e-f708-4173-b225-138a3dbc2260 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:08.875049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91786ce-7836-437b-9273-116798b19da0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:08.919547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee6883-af8e-45a3-82f0-e883f11c22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:08.920563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6883-af8e-45a3-82f0-e883f11c22a1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.921189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6883-af8e-45a3-82f0-e883f11c22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.921833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6883-af8e-45a3-82f0-e883f11c22a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.928047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1c21e-f708-4173-b225-138a3dbc2260 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:08.930168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:08.938934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee6883-af8e-45a3-82f0-e883f11c22a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-fdee6883-af8e-45a3-82f0-e883f11c22a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:08.946650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:08.947642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:08.948914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:08.955389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 156/304] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:08.955389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee6883-af8e-45a3-82f0-e883f11c22a1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:08.967622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4807a-854e-4b2e-b44d-f6c77b842714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:08.968271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4807a-854e-4b2e-b44d-f6c77b842714 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:08.968545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4807a-854e-4b2e-b44d-f6c77b842714 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:08.968814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4807a-854e-4b2e-b44d-f6c77b842714 None admin] PATH_INFO: `/v3/system/users/3275a7332f044e26a4791a02f1810a57/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:08.970285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4807a-854e-4b2e-b44d-f6c77b842714 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3275a7332f044e26a4791a02f1810a57, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:08.971641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4807a-854e-4b2e-b44d-f6c77b842714 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:08.981569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 149/305] 10.222.0.15 () {58 vars in 1334 bytes} [Fri Sep 9 01:35:08 2022] PUT /identity/v3/system/users/3275a7332f044e26a4791a02f1810a57/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:35:08.986221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4807a-854e-4b2e-b44d-f6c77b842714 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:08.995601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:09.100074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee6883-af8e-45a3-82f0-e883f11c22a1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:09.123947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4807a-854e-4b2e-b44d-f6c77b842714 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:09.135350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436e29d-2171-4e7c-9938-ae356fec47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:09.135907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436e29d-2171-4e7c-9938-ae356fec47ff None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.136188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436e29d-2171-4e7c-9938-ae356fec47ff None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.136464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436e29d-2171-4e7c-9938-ae356fec47ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.137655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436e29d-2171-4e7c-9938-ae356fec47ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:09.139152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436e29d-2171-4e7c-9938-ae356fec47ff None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:09.140996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 157/306] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:09.146781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed19785-6fd9-4589-a829-51ee4ff59478 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.147062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed19785-6fd9-4589-a829-51ee4ff59478 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.147339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed19785-6fd9-4589-a829-51ee4ff59478 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.148851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 158/307] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:09.149421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed19785-6fd9-4589-a829-51ee4ff59478 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:09.169397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b34a13-ca6d-4028-a344-660f4a014bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:09.170276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b34a13-ca6d-4028-a344-660f4a014bad None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.170700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b34a13-ca6d-4028-a344-660f4a014bad None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.171253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b34a13-ca6d-4028-a344-660f4a014bad None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.172192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b34a13-ca6d-4028-a344-660f4a014bad None admin] RBAC: Adding query filter params (name=reader) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:09.172647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b34a13-ca6d-4028-a344-660f4a014bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:09.174367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b34a13-ca6d-4028-a344-660f4a014bad None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:09.183764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 150/308] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:09.185387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734bad8-167b-416c-9133-e074c36d5988 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.185836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734bad8-167b-416c-9133-e074c36d5988 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.186255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734bad8-167b-416c-9133-e074c36d5988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.206669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9734bad8-167b-416c-9133-e074c36d5988 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:09.207852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9734bad8-167b-416c-9133-e074c36d5988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:09.233029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:09.262444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:09.267568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed19785-6fd9-4589-a829-51ee4ff59478 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:09.280405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 151/309] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:09.281397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734bad8-167b-416c-9133-e074c36d5988 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:09.313233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d13f0-b62e-42d2-92eb-230a12a44e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:09.314157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d13f0-b62e-42d2-92eb-230a12a44e67 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.314604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d13f0-b62e-42d2-92eb-230a12a44e67 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.315041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d13f0-b62e-42d2-92eb-230a12a44e67 None admin] PATH_INFO: `/v3/groups` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.316001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d13f0-b62e-42d2-92eb-230a12a44e67 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:09.317724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d13f0-b62e-42d2-92eb-230a12a44e67 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:09.339045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 159/310] 10.222.0.15 () {60 vars in 1134 bytes} [Fri Sep 9 01:35:09 2022] POST /identity/v3/groups => generated 229 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:35:09.410441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734bad8-167b-416c-9133-e074c36d5988 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:09.462895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fa81e-9284-49d0-ba3c-140a9b437cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:09.464517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa81e-9284-49d0-ba3c-140a9b437cea None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.465012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa81e-9284-49d0-ba3c-140a9b437cea None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.465527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fa81e-9284-49d0-ba3c-140a9b437cea None admin] PATH_INFO: `/v3/users/bf26a96dd0284b7da9ed7958ee16c4e0` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.486094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fa81e-9284-49d0-ba3c-140a9b437cea None admin] RBAC: Authorizing `identity:get_user(user_id=bf26a96dd0284b7da9ed7958ee16c4e0)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:09.487655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697fa81e-9284-49d0-ba3c-140a9b437cea None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:09.489858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 152/311] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:09 2022] GET /identity/v3/users/bf26a96dd0284b7da9ed7958ee16c4e0 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:09.495670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d13f0-b62e-42d2-92eb-230a12a44e67 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:09.593159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fa81e-9284-49d0-ba3c-140a9b437cea None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:09.611824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:09.612142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d13f0-b62e-42d2-92eb-230a12a44e67 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:09.648771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:09.649505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.649812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.650150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.650936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] RBAC: Adding query filter params (role.id=8041ef90cc68470fa80216023a3091c8, user.id=bf26a96dd0284b7da9ed7958ee16c4e0, scope.system=all) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:09.651245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:09.652779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:09.658685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 160/312] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 9 01:35:09 2022] GET /identity/v3/role_assignments?user.id=bf26a96dd0284b7da9ed7958ee16c4e0&scope.system=all&role.id=8041ef90cc68470fa80216023a3091c8 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:09.706039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697fa81e-9284-49d0-ba3c-140a9b437cea None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:09.744272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52ffb7-1de4-47b1-8a4f-efd8a5a314da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:09.744925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52ffb7-1de4-47b1-8a4f-efd8a5a314da None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.745233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52ffb7-1de4-47b1-8a4f-efd8a5a314da None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.745502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52ffb7-1de4-47b1-8a4f-efd8a5a314da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.757536 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52ffb7-1de4-47b1-8a4f-efd8a5a314da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-5a52ffb7-1de4-47b1-8a4f-efd8a5a314da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:09.763300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:09.764205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:09.764960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 153/313] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:09.770625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:09.798238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:09.880457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf542c-d9b0-4617-a42a-f649bc94d2cf None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:09.918000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:09.918970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:09.919424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:09.919843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:09.920684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:09.921165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:09.923964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:09.934023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 161/314] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:09.945118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52ffb7-1de4-47b1-8a4f-efd8a5a314da None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:10.019242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:10.043413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52ffb7-1de4-47b1-8a4f-efd8a5a314da None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:10.076953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80217-47ff-4ca4-a458-3eb3a88d96fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:10.077615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80217-47ff-4ca4-a458-3eb3a88d96fb None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:10.077884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80217-47ff-4ca4-a458-3eb3a88d96fb None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:10.078236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d80217-47ff-4ca4-a458-3eb3a88d96fb None admin] PATH_INFO: `/v3/users/e426f095aff24f048868e266239a8ada` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:10.079357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d80217-47ff-4ca4-a458-3eb3a88d96fb None admin] RBAC: Authorizing `identity:get_user(user_id=e426f095aff24f048868e266239a8ada)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:10.080778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d80217-47ff-4ca4-a458-3eb3a88d96fb None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:10.082491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 154/315] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:09 2022] GET /identity/v3/users/e426f095aff24f048868e266239a8ada => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:10.088473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:10.175741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:10.197864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf1c44-733c-4466-b3e7-ee45c5e979ea None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:10.233490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0396e-10c4-450d-909c-25f9c8ce3086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:10.234100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0396e-10c4-450d-909c-25f9c8ce3086 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:10.234378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0396e-10c4-450d-909c-25f9c8ce3086 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:10.234650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0396e-10c4-450d-909c-25f9c8ce3086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:10.304335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0396e-10c4-450d-909c-25f9c8ce3086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:10.305744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0396e-10c4-450d-909c-25f9c8ce3086 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:10.406183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:10.409687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 162/316] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:10.415395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80217-47ff-4ca4-a458-3eb3a88d96fb None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:10.520805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d80217-47ff-4ca4-a458-3eb3a88d96fb None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:10.554913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab405ed-8a16-41a2-928e-aefd1bbad2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:10.555465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab405ed-8a16-41a2-928e-aefd1bbad2d7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:10.555737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab405ed-8a16-41a2-928e-aefd1bbad2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:10.556001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab405ed-8a16-41a2-928e-aefd1bbad2d7 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:10.557787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab405ed-8a16-41a2-928e-aefd1bbad2d7 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:10.559199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab405ed-8a16-41a2-928e-aefd1bbad2d7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:10.561128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 155/317] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:10 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:10.566702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0396e-10c4-450d-909c-25f9c8ce3086 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:10.670988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0396e-10c4-450d-909c-25f9c8ce3086 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:10.706218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c8411-1a7d-4eb7-b2ea-5592dd68330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:10.706824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8411-1a7d-4eb7-b2ea-5592dd68330a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:10.707098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8411-1a7d-4eb7-b2ea-5592dd68330a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:10.707368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c8411-1a7d-4eb7-b2ea-5592dd68330a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:10.708096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c8411-1a7d-4eb7-b2ea-5592dd68330a None admin] RBAC: Adding query filter params (role.id=8041ef90cc68470fa80216023a3091c8, scope.project.id=5a9037a124a2489992eba863bf5e9b44, user.id=e426f095aff24f048868e266239a8ada) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:10.708400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c8411-1a7d-4eb7-b2ea-5592dd68330a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:10.709871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c8411-1a7d-4eb7-b2ea-5592dd68330a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:10.716071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 163/318] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:10 2022] GET /identity/v3/role_assignments?user.id=e426f095aff24f048868e266239a8ada&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=8041ef90cc68470fa80216023a3091c8 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:11.235404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462d9ea-146e-4690-9ae5-c15c59b61478 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:11.235893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462d9ea-146e-4690-9ae5-c15c59b61478 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:11.235893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462d9ea-146e-4690-9ae5-c15c59b61478 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:11.237264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 156/319] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:11.242316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6671f63-0b7c-43ec-8b95-1db217c40e47 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:11.242692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6671f63-0b7c-43ec-8b95-1db217c40e47 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:11.242857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6671f63-0b7c-43ec-8b95-1db217c40e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:11.261584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f6671f63-0b7c-43ec-8b95-1db217c40e47 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:11.262756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f6671f63-0b7c-43ec-8b95-1db217c40e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:11.308324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:11.332270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 164/320] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:11.597954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462d9ea-146e-4690-9ae5-c15c59b61478 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:11.614516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:11.656466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea004-7bac-4b03-8934-13bd07d038ea None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:11.656740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea004-7bac-4b03-8934-13bd07d038ea None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:11.657211 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea004-7bac-4b03-8934-13bd07d038ea None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:11.658548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 165/321] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:11.663398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc56b0-6036-4fcd-8942-072ba68728e7 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:11.663868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc56b0-6036-4fcd-8942-072ba68728e7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:11.664135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc56b0-6036-4fcd-8942-072ba68728e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:11.685008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0adc56b0-6036-4fcd-8942-072ba68728e7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:11.686213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0adc56b0-6036-4fcd-8942-072ba68728e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:11.704959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462d9ea-146e-4690-9ae5-c15c59b61478 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:11.740907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ce50b-d17f-4074-bdf6-56f1137d46d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:11.741555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ce50b-d17f-4074-bdf6-56f1137d46d3 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:11.741830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ce50b-d17f-4074-bdf6-56f1137d46d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:11.742095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ce50b-d17f-4074-bdf6-56f1137d46d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:11.753275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ce50b-d17f-4074-bdf6-56f1137d46d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:11.756513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 166/322] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-207ce50b-d17f-4074-bdf6-56f1137d46d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:11.758336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:11.759256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:11.765161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 157/323] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:11.765815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc56b0-6036-4fcd-8942-072ba68728e7 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:11.868415 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc56b0-6036-4fcd-8942-072ba68728e7 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:11.878086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:11.907278 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:11.907886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:11.908165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:11.908432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:11.909110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:11.909415 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:11.910800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:11.918358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d0afd-801c-4046-9754-affcd99ff4f7 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:11.918358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d0afd-801c-4046-9754-affcd99ff4f7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:11.918358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d0afd-801c-4046-9754-affcd99ff4f7 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:11.918358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 158/324] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:35:11.923527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f5188-bd69-4c14-8072-e2463d848e4d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:11.923813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f5188-bd69-4c14-8072-e2463d848e4d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:11.924082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f5188-bd69-4c14-8072-e2463d848e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:11.925201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 167/325] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:11.926866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:11.944998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1a4f5188-bd69-4c14-8072-e2463d848e4d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:11.946207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1a4f5188-bd69-4c14-8072-e2463d848e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:12.021380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 159/326] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:12.022661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f5188-bd69-4c14-8072-e2463d848e4d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:12.025734 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:12.036341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913919b-6dff-499b-a75d-aed6a50b3577 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:12.075302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4066a9-3a98-4bcf-80f4-c80e460b7adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:12.076092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4066a9-3a98-4bcf-80f4-c80e460b7adc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:12.076639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4066a9-3a98-4bcf-80f4-c80e460b7adc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:12.076923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4066a9-3a98-4bcf-80f4-c80e460b7adc None admin] PATH_INFO: `/v3/users/3275a7332f044e26a4791a02f1810a57` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:12.078079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4066a9-3a98-4bcf-80f4-c80e460b7adc None admin] RBAC: Authorizing `identity:get_user(user_id=3275a7332f044e26a4791a02f1810a57)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:12.079942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4066a9-3a98-4bcf-80f4-c80e460b7adc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:12.085200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 168/327] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:11 2022] GET /identity/v3/users/3275a7332f044e26a4791a02f1810a57 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:12.087619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4066a9-3a98-4bcf-80f4-c80e460b7adc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:12.127311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:12.135548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f5188-bd69-4c14-8072-e2463d848e4d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:12.174950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6e564-e744-4ee4-a993-dca0c617f711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:12.175571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6e564-e744-4ee4-a993-dca0c617f711 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:12.175843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6e564-e744-4ee4-a993-dca0c617f711 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:12.176107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6e564-e744-4ee4-a993-dca0c617f711 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:12.177213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6e564-e744-4ee4-a993-dca0c617f711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:12.178647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6e564-e744-4ee4-a993-dca0c617f711 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:12.181116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 160/328] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:12.185563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6e564-e744-4ee4-a993-dca0c617f711 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:12.191543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4066a9-3a98-4bcf-80f4-c80e460b7adc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:12.228318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:12.228984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:12.229298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:12.229573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:12.230678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] RBAC: Adding query filter params (role.id=18bebd2151824cdfa386be482ff8114b, user.id=3275a7332f044e26a4791a02f1810a57, scope.system=all) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:12.230988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:12.232994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:12.248532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 169/329] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 9 01:35:12 2022] GET /identity/v3/role_assignments?user.id=3275a7332f044e26a4791a02f1810a57&scope.system=all&role.id=18bebd2151824cdfa386be482ff8114b => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:12.263465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:12.286363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:12.299041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6e564-e744-4ee4-a993-dca0c617f711 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:12.338340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e529bce-6c1c-4b44-baf1-7e70219d4456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:12.338957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e529bce-6c1c-4b44-baf1-7e70219d4456 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:12.339237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e529bce-6c1c-4b44-baf1-7e70219d4456 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:12.339507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e529bce-6c1c-4b44-baf1-7e70219d4456 None admin] PATH_INFO: `/v3/groups` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:12.340153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e529bce-6c1c-4b44-baf1-7e70219d4456 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:12.342444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e529bce-6c1c-4b44-baf1-7e70219d4456 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:12.373101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 161/330] 10.222.0.15 () {60 vars in 1134 bytes} [Fri Sep 9 01:35:12 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:35:12.406414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:12.414685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb3a43-6af6-4fa1-857c-bc9984b5aeef None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:12.456938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebebc8e-3e8f-4994-aa35-1187b5ee081d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:12.457648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebebc8e-3e8f-4994-aa35-1187b5ee081d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:12.457935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebebc8e-3e8f-4994-aa35-1187b5ee081d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:12.458206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebebc8e-3e8f-4994-aa35-1187b5ee081d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:12.471075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebebc8e-3e8f-4994-aa35-1187b5ee081d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-eebebc8e-3e8f-4994-aa35-1187b5ee081d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:12.477053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:12.478134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:12.479462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 170/331] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:12.484944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e529bce-6c1c-4b44-baf1-7e70219d4456 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:12.590633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e529bce-6c1c-4b44-baf1-7e70219d4456 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:12.618151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:12.626446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:12.627058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:12.627335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:12.627607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:12.628273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:12.628630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:12.629968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:12.639331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 162/332] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:12.645151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebebc8e-3e8f-4994-aa35-1187b5ee081d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:12.751715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebebc8e-3e8f-4994-aa35-1187b5ee081d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:12.789558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f04848-7ecf-4582-afab-76bddc1c7309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:12.790204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f04848-7ecf-4582-afab-76bddc1c7309 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:12.790480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f04848-7ecf-4582-afab-76bddc1c7309 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:12.790747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f04848-7ecf-4582-afab-76bddc1c7309 None admin] PATH_INFO: `/v3/users/bf26a96dd0284b7da9ed7958ee16c4e0` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:12.791983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f04848-7ecf-4582-afab-76bddc1c7309 None admin] RBAC: Authorizing `identity:get_user(user_id=bf26a96dd0284b7da9ed7958ee16c4e0)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:12.793458 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f04848-7ecf-4582-afab-76bddc1c7309 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:12.795349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 171/333] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:12 2022] GET /identity/v3/users/bf26a96dd0284b7da9ed7958ee16c4e0 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:12.800817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:12.912346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d8e53-c405-4ea5-853b-2b209e22de97 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:12.955233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f1369-85b3-40fc-81bf-1b9daaa1bc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:12.956265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1369-85b3-40fc-81bf-1b9daaa1bc77 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:12.956544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1369-85b3-40fc-81bf-1b9daaa1bc77 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:12.956815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f1369-85b3-40fc-81bf-1b9daaa1bc77 None admin] PATH_INFO: `/v3/system/users/bf26a96dd0284b7da9ed7958ee16c4e0/roles/8041ef90cc68470fa80216023a3091c8` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:12.958336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f1369-85b3-40fc-81bf-1b9daaa1bc77 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bf26a96dd0284b7da9ed7958ee16c4e0, role_id=8041ef90cc68470fa80216023a3091c8)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:12.959720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f1369-85b3-40fc-81bf-1b9daaa1bc77 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:12.969006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 163/334] 10.222.0.15 () {58 vars in 1334 bytes} [Fri Sep 9 01:35:12 2022] PUT /identity/v3/system/users/bf26a96dd0284b7da9ed7958ee16c4e0/roles/8041ef90cc68470fa80216023a3091c8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:15.157165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf893b-19ae-47bd-b151-63598d955a32 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:15.157646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf893b-19ae-47bd-b151-63598d955a32 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:15.157646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf893b-19ae-47bd-b151-63598d955a32 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:15.159253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 172/335] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:15.165916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465d9e8-eb78-4912-a2b7-9fe19abff78d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:15.166883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465d9e8-eb78-4912-a2b7-9fe19abff78d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:15.167367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465d9e8-eb78-4912-a2b7-9fe19abff78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:15.187777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7465d9e8-eb78-4912-a2b7-9fe19abff78d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:15.189359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7465d9e8-eb78-4912-a2b7-9fe19abff78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:15.247105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:15.265253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 164/336] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:15.435676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545569f2-c25d-4743-b4a7-ceed29d7394c None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:15.436281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545569f2-c25d-4743-b4a7-ceed29d7394c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:15.436791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545569f2-c25d-4743-b4a7-ceed29d7394c None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:15.438456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 173/337] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:15.444165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b270747-8088-497c-a6b0-c78e8fe10d01 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:15.444796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b270747-8088-497c-a6b0-c78e8fe10d01 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:15.445315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b270747-8088-497c-a6b0-c78e8fe10d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:15.474284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9b270747-8088-497c-a6b0-c78e8fe10d01 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:15.475916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9b270747-8088-497c-a6b0-c78e8fe10d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:15.561804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 165/338] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:15.589443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974366c7-00f8-4f5f-bf72-6e6b54aa862a None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:15.590046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974366c7-00f8-4f5f-bf72-6e6b54aa862a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:15.590543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974366c7-00f8-4f5f-bf72-6e6b54aa862a None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:15.592080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 174/339] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:15.597471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a4f0e-77c0-4aed-986f-c99858a0976f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:15.598024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a4f0e-77c0-4aed-986f-c99858a0976f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:15.598492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a4f0e-77c0-4aed-986f-c99858a0976f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:15.622948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c04a4f0e-77c0-4aed-986f-c99858a0976f None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:15.624455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c04a4f0e-77c0-4aed-986f-c99858a0976f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:15.703503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 166/340] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:15.770174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974366c7-00f8-4f5f-bf72-6e6b54aa862a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:15.798508 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:15.902698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974366c7-00f8-4f5f-bf72-6e6b54aa862a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:15.950963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c189d1-ab2f-41d3-9951-2764962f7480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:15.951594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c189d1-ab2f-41d3-9951-2764962f7480 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:15.951867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c189d1-ab2f-41d3-9951-2764962f7480 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:15.952135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c189d1-ab2f-41d3-9951-2764962f7480 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:15.963243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c189d1-ab2f-41d3-9951-2764962f7480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:15.967312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a4f0e-77c0-4aed-986f-c99858a0976f None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-f7c189d1-ab2f-41d3-9951-2764962f7480 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:15.975715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:15.976590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:15.977389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 175/341] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:15.986309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c189d1-ab2f-41d3-9951-2764962f7480 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:16.026539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:16.093322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a4f0e-77c0-4aed-986f-c99858a0976f None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:16.133031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42821e-4400-413b-8c9b-4df88ae3d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:16.133705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42821e-4400-413b-8c9b-4df88ae3d63a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.133982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42821e-4400-413b-8c9b-4df88ae3d63a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.135199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42821e-4400-413b-8c9b-4df88ae3d63a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.149342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42821e-4400-413b-8c9b-4df88ae3d63a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:16.149536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-2f42821e-4400-413b-8c9b-4df88ae3d63a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:16.157100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:16.158038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 167/342] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:35:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:16.158799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42821e-4400-413b-8c9b-4df88ae3d63a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:16.184478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c189d1-ab2f-41d3-9951-2764962f7480 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:16.258547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:16.259085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.259361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.259669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.260429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:16.260736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:16.262270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:16.267424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42821e-4400-413b-8c9b-4df88ae3d63a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:16.272483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 176/343] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:16.273380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:16.275779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:16.310681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf374e-8d15-45e5-9301-de76065597a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:16.311326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf374e-8d15-45e5-9301-de76065597a2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.311749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf374e-8d15-45e5-9301-de76065597a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.312149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf374e-8d15-45e5-9301-de76065597a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.323298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf374e-8d15-45e5-9301-de76065597a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-1eaf374e-8d15-45e5-9301-de76065597a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:16.328766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:16.329705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:16.331090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 168/344] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:16.331735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bdf81-2cef-4ec6-b9cd-89c1ad38bb64 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.331735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bdf81-2cef-4ec6-b9cd-89c1ad38bb64 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.331859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bdf81-2cef-4ec6-b9cd-89c1ad38bb64 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.333983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 169/345] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:16.334691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bdf81-2cef-4ec6-b9cd-89c1ad38bb64 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:16.397600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de5f31-558b-4537-9a41-bd0dccbb9267 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:16.435850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:16.436465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.436745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.437017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.437757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:16.438061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:16.439469 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:16.453904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 177/346] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:16.455071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:16.477484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bdf81-2cef-4ec6-b9cd-89c1ad38bb64 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:16.481437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:16.525957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963aea2-c007-49d7-83b5-bd773f679819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:16.526485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963aea2-c007-49d7-83b5-bd773f679819 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.526762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963aea2-c007-49d7-83b5-bd773f679819 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.527032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963aea2-c007-49d7-83b5-bd773f679819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.570074 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:16.577446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190d3dc-5b0d-49bd-a52a-86de560a08ab None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:16.610199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963aea2-c007-49d7-83b5-bd773f679819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:16.611972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963aea2-c007-49d7-83b5-bd773f679819 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:16.620506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af13d3-7d6e-4fc5-a11e-8390b7238385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:16.621427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af13d3-7d6e-4fc5-a11e-8390b7238385 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.621903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af13d3-7d6e-4fc5-a11e-8390b7238385 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.622342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af13d3-7d6e-4fc5-a11e-8390b7238385 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.623605 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af13d3-7d6e-4fc5-a11e-8390b7238385 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:16.624099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af13d3-7d6e-4fc5-a11e-8390b7238385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:16.625658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af13d3-7d6e-4fc5-a11e-8390b7238385 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:16.636751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 178/347] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:16.638188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4f49d-79e6-4a8c-8c4b-81c1da1bd0a3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.638617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4f49d-79e6-4a8c-8c4b-81c1da1bd0a3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.638884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4f49d-79e6-4a8c-8c4b-81c1da1bd0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.659350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6da4f49d-79e6-4a8c-8c4b-81c1da1bd0a3 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:16.660536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6da4f49d-79e6-4a8c-8c4b-81c1da1bd0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:16.740244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:16.745171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 170/348] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:16.745171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963aea2-c007-49d7-83b5-bd773f679819 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:16.755603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 179/349] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:16.756953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88849b80-30c9-40d7-b785-cc02ae9757ba None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.757261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88849b80-30c9-40d7-b785-cc02ae9757ba None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.757670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88849b80-30c9-40d7-b785-cc02ae9757ba None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.759120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 180/350] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:16.759896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88849b80-30c9-40d7-b785-cc02ae9757ba None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:16.833261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:16.879324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963aea2-c007-49d7-83b5-bd773f679819 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:16.882759 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88849b80-30c9-40d7-b785-cc02ae9757ba None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:16.932993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a7c0a-8a24-4100-8270-cc3726911477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:16.933820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a7c0a-8a24-4100-8270-cc3726911477 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.933925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a7c0a-8a24-4100-8270-cc3726911477 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.934400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a7c0a-8a24-4100-8270-cc3726911477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.943620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0adcf5-8bfe-429a-bdd3-c08a059f486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:16.944566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0adcf5-8bfe-429a-bdd3-c08a059f486c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:16.945041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0adcf5-8bfe-429a-bdd3-c08a059f486c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:16.945548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0adcf5-8bfe-429a-bdd3-c08a059f486c None admin] PATH_INFO: `/v3/users/bf26a96dd0284b7da9ed7958ee16c4e0` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:16.948055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0adcf5-8bfe-429a-bdd3-c08a059f486c None admin] RBAC: Authorizing `identity:get_user(user_id=bf26a96dd0284b7da9ed7958ee16c4e0)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:16.950203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0adcf5-8bfe-429a-bdd3-c08a059f486c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:16.952828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 181/351] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:35:16 2022] GET /identity/v3/users/bf26a96dd0284b7da9ed7958ee16c4e0 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:16.953853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0adcf5-8bfe-429a-bdd3-c08a059f486c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.026776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a7c0a-8a24-4100-8270-cc3726911477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:17.029423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a7c0a-8a24-4100-8270-cc3726911477 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:17.041981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:17.092542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0adcf5-8bfe-429a-bdd3-c08a059f486c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:17.126716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:17.139564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd2147-ac37-4ffd-83df-4cdbdb627e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:17.140201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd2147-ac37-4ffd-83df-4cdbdb627e08 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.140478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd2147-ac37-4ffd-83df-4cdbdb627e08 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.140747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd2147-ac37-4ffd-83df-4cdbdb627e08 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.141919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd2147-ac37-4ffd-83df-4cdbdb627e08 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:17.143343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd2147-ac37-4ffd-83df-4cdbdb627e08 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:17.145186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 182/352] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:16 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:17.146837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02edc49-d664-4804-a4a1-d4e17c0a6028 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.147127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02edc49-d664-4804-a4a1-d4e17c0a6028 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.147398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02edc49-d664-4804-a4a1-d4e17c0a6028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.167065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d02edc49-d664-4804-a4a1-d4e17c0a6028 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:17.168205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d02edc49-d664-4804-a4a1-d4e17c0a6028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:17.169550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 171/353] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:17.170238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a7c0a-8a24-4100-8270-cc3726911477 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.257212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 183/354] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:17.257729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02edc49-d664-4804-a4a1-d4e17c0a6028 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.310333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a7c0a-8a24-4100-8270-cc3726911477 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:17.336236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:17.363510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:17.364728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.365383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.365833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.367065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] RBAC: Adding query filter params (role.id=8041ef90cc68470fa80216023a3091c8, user.id=bf26a96dd0284b7da9ed7958ee16c4e0, scope.system=all) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:17.367548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:17.370054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:17.377658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02edc49-d664-4804-a4a1-d4e17c0a6028 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:17.385686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 172/355] 10.222.0.15 () {56 vars in 1308 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/role_assignments?user.id=bf26a96dd0284b7da9ed7958ee16c4e0&scope.system=all&role.id=8041ef90cc68470fa80216023a3091c8 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:17.387180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.438937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14513a-772e-42a4-b57b-1a7be1d9a6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:17.439590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14513a-772e-42a4-b57b-1a7be1d9a6f6 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.439864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14513a-772e-42a4-b57b-1a7be1d9a6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.440129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14513a-772e-42a4-b57b-1a7be1d9a6f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.447039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:17.461252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14513a-772e-42a4-b57b-1a7be1d9a6f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-7e14513a-772e-42a4-b57b-1a7be1d9a6f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:17.468896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:17.469870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:17.470625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 184/356] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:17.471262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14513a-772e-42a4-b57b-1a7be1d9a6f6 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.554960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6bd9b-ca33-4c49-a16d-a551f61770c9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:17.595583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ab737-4c9d-45dc-8aa7-3d378d795b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:17.600206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ab737-4c9d-45dc-8aa7-3d378d795b2f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.600206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ab737-4c9d-45dc-8aa7-3d378d795b2f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.600206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ab737-4c9d-45dc-8aa7-3d378d795b2f None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.602187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14513a-772e-42a4-b57b-1a7be1d9a6f6 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:17.605513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ab737-4c9d-45dc-8aa7-3d378d795b2f None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:17.608143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ab737-4c9d-45dc-8aa7-3d378d795b2f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:17.609933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 173/357] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:17.611143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bd52a-aa09-449c-a022-cba3ed02ecd6 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.611418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bd52a-aa09-449c-a022-cba3ed02ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.611683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bd52a-aa09-449c-a022-cba3ed02ecd6 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.613832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 174/358] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:17.614574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bd52a-aa09-449c-a022-cba3ed02ecd6 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.645665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:17.654746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe1e2e-c91a-43d9-86e5-9799fcaf734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:17.655375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1e2e-c91a-43d9-86e5-9799fcaf734f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.655647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1e2e-c91a-43d9-86e5-9799fcaf734f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.655914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1e2e-c91a-43d9-86e5-9799fcaf734f None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.657574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe1e2e-c91a-43d9-86e5-9799fcaf734f None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:17.659023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe1e2e-c91a-43d9-86e5-9799fcaf734f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:17.661493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 185/359] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:17.662264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe1e2e-c91a-43d9-86e5-9799fcaf734f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.734916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:17.753664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bd52a-aa09-449c-a022-cba3ed02ecd6 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:17.789979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:17.790597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.791415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.791691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.792357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] RBAC: Adding query filter params (name=member) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:17.792662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:17.794081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:17.794271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe1e2e-c91a-43d9-86e5-9799fcaf734f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:17.806299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 175/360] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:17.807108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.832343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f86398-8d29-45ab-905b-f37ea027df1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:17.832974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f86398-8d29-45ab-905b-f37ea027df1a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.833276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f86398-8d29-45ab-905b-f37ea027df1a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.833541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f86398-8d29-45ab-905b-f37ea027df1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.844644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f86398-8d29-45ab-905b-f37ea027df1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-c9f86398-8d29-45ab-905b-f37ea027df1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:17.849791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:17.850739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:17.851612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 186/361] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:17.852348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ce41e-890b-4d80-96c1-449038541586 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.852628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ce41e-890b-4d80-96c1-449038541586 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.852912 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ce41e-890b-4d80-96c1-449038541586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.876138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f71ce41e-890b-4d80-96c1-449038541586 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:17.877325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f71ce41e-890b-4d80-96c1-449038541586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:17.926388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc116c-2522-4419-905b-38ffa5788919 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:17.937531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:17.958027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 187/362] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:17.958871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ce41e-890b-4d80-96c1-449038541586 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.969765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:17.971923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:17.972558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:17.972847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:17.973915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] RBAC: Adding query filter params (group.id=ec73010a7d3344249232dd06d49f2b9f, role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=b6271cacfed747da992ff32e5f4813f9) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:17.974240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:17.975839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:17.982197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 176/363] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/role_assignments?group.id=ec73010a7d3344249232dd06d49f2b9f&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=18bebd2151824cdfa386be482ff8114b => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:17.983000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:17.999309 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:18.109049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ce41e-890b-4d80-96c1-449038541586 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:18.118808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8d8e6-bb75-44b9-9172-211fd117c602 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:18.161719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f43fc-a2c9-48c8-9eac-cc1a215a2ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:18.162284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f43fc-a2c9-48c8-9eac-cc1a215a2ada None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:18.162578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f43fc-a2c9-48c8-9eac-cc1a215a2ada None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:18.163191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f43fc-a2c9-48c8-9eac-cc1a215a2ada None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:18.164559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f43fc-a2c9-48c8-9eac-cc1a215a2ada None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:18.166224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f43fc-a2c9-48c8-9eac-cc1a215a2ada None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:18.168193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 188/364] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:18.168618 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe36b6-e710-478d-9ec9-c5b217b334ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:18.168980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f43fc-a2c9-48c8-9eac-cc1a215a2ada None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:18.169710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe36b6-e710-478d-9ec9-c5b217b334ca None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:18.170206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe36b6-e710-478d-9ec9-c5b217b334ca None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:18.178343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe36b6-e710-478d-9ec9-c5b217b334ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:18.250154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:18.263479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe36b6-e710-478d-9ec9-c5b217b334ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:18.265078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe36b6-e710-478d-9ec9-c5b217b334ca None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:18.283651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:18.308455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f43fc-a2c9-48c8-9eac-cc1a215a2ada None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:18.358283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:18.358283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:18.358587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:18.358722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:18.359435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:18.359737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:18.361103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:18.373358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 189/365] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:18.374201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:18.395405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 177/366] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:18.397950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe36b6-e710-478d-9ec9-c5b217b334ca None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:18.510332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a038c6-d2e9-4185-9ebf-e74e4d2b9670 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:18.531576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:18.537323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe36b6-e710-478d-9ec9-c5b217b334ca None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:18.557041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:18.557752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:18.558028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:18.558297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:18.559021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] RBAC: Adding query filter params (group.id=ec73010a7d3344249232dd06d49f2b9f, role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=b6271cacfed747da992ff32e5f4813f9) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:18.559330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:18.559976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:18.560803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:18.569086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 190/367] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:18 2022] GET /identity/v3/role_assignments?group.id=ec73010a7d3344249232dd06d49f2b9f&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=eb7bedf01e0744eeb26681978db4121c => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:18.569086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:18.588492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f4ad7-d120-4b9d-92db-cc27ac561553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:18.589299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f4ad7-d120-4b9d-92db-cc27ac561553 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:18.589651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f4ad7-d120-4b9d-92db-cc27ac561553 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:18.590021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f4ad7-d120-4b9d-92db-cc27ac561553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:18.603350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f4ad7-d120-4b9d-92db-cc27ac561553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-f54f4ad7-d120-4b9d-92db-cc27ac561553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:18.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:18.610452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:18.611561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 178/368] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:35:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:18.617672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f4ad7-d120-4b9d-92db-cc27ac561553 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:18.719339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361312e-2f14-46f5-9258-ecc8380fbd01 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:18.762074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909962b-da32-4fc3-935c-3477a2d8b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:18.762616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909962b-da32-4fc3-935c-3477a2d8b8ff None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:18.762893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909962b-da32-4fc3-935c-3477a2d8b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:18.763165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909962b-da32-4fc3-935c-3477a2d8b8ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:18.767789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f4ad7-d120-4b9d-92db-cc27ac561553 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:18.819843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75118c51-06d9-4443-a31c-89033e9cb95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:18.820478 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118c51-06d9-4443-a31c-89033e9cb95b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:18.820746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118c51-06d9-4443-a31c-89033e9cb95b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:18.821006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75118c51-06d9-4443-a31c-89033e9cb95b None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:18.824084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:18.825128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75118c51-06d9-4443-a31c-89033e9cb95b None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:18.826701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75118c51-06d9-4443-a31c-89033e9cb95b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:18.833012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 179/369] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:18 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:18.833012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75118c51-06d9-4443-a31c-89033e9cb95b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:18.865889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909962b-da32-4fc3-935c-3477a2d8b8ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:18.867345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2909962b-da32-4fc3-935c-3477a2d8b8ff None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:18.885560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:18.965338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75118c51-06d9-4443-a31c-89033e9cb95b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:18.994529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 191/370] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:18.995535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909962b-da32-4fc3-935c-3477a2d8b8ff None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:19.013992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.015239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.015526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.015798 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.016764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:19.017087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.019144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.030981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 180/371] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:19.032687 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:19.100091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:19.128913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2909962b-da32-4fc3-935c-3477a2d8b8ff None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:19.170894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a8404-ef5c-412f-a82f-1b5da21d8190 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:19.173810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:19.182645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f932df5-ad56-4aa9-b0c4-2a989306fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.183595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f932df5-ad56-4aa9-b0c4-2a989306fc3b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.184396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f932df5-ad56-4aa9-b0c4-2a989306fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.185253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f932df5-ad56-4aa9-b0c4-2a989306fc3b None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.186777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f932df5-ad56-4aa9-b0c4-2a989306fc3b None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.189142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f932df5-ad56-4aa9-b0c4-2a989306fc3b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.191408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 192/372] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:18 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:19.193075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f932df5-ad56-4aa9-b0c4-2a989306fc3b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:19.212353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda10a4-dd3d-4aa4-a509-0a88bedb0bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.213123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda10a4-dd3d-4aa4-a509-0a88bedb0bad None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.213570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda10a4-dd3d-4aa4-a509-0a88bedb0bad None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.213843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda10a4-dd3d-4aa4-a509-0a88bedb0bad None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.215138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda10a4-dd3d-4aa4-a509-0a88bedb0bad None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.216570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda10a4-dd3d-4aa4-a509-0a88bedb0bad None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.218527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 181/373] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:19 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:19.219526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda10a4-dd3d-4aa4-a509-0a88bedb0bad None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:19.323005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f932df5-ad56-4aa9-b0c4-2a989306fc3b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:19.333493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda10a4-dd3d-4aa4-a509-0a88bedb0bad None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:19.368738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684c7f5-cbd6-4856-b651-abe7fb429734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.369909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684c7f5-cbd6-4856-b651-abe7fb429734 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.370387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684c7f5-cbd6-4856-b651-abe7fb429734 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.370490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.370965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684c7f5-cbd6-4856-b651-abe7fb429734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.371157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.371403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.371670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.372510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] RBAC: Adding query filter params (group.id=ec73010a7d3344249232dd06d49f2b9f, role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=5a9037a124a2489992eba863bf5e9b44) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:19.372840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.374271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.381260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 182/374] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:19 2022] GET /identity/v3/role_assignments?group.id=ec73010a7d3344249232dd06d49f2b9f&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=18bebd2151824cdfa386be482ff8114b => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:19.382078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:19.392854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:19.427630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:19.465911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684c7f5-cbd6-4856-b651-abe7fb429734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.468523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684c7f5-cbd6-4856-b651-abe7fb429734 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.512766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14758516-f7d3-4730-92e0-fe6db8cfd999 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:19.554809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c31ef-1b35-4115-a508-ad9f474065b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.555453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c31ef-1b35-4115-a508-ad9f474065b7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.555726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c31ef-1b35-4115-a508-ad9f474065b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.556003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c31ef-1b35-4115-a508-ad9f474065b7 None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.558998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c31ef-1b35-4115-a508-ad9f474065b7 None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.560601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c31ef-1b35-4115-a508-ad9f474065b7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.562303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 183/375] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:19 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:19.568489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c31ef-1b35-4115-a508-ad9f474065b7 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:19.605457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 193/376] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:19.612480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684c7f5-cbd6-4856-b651-abe7fb429734 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:19.670795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:19.704549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c31ef-1b35-4115-a508-ad9f474065b7 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:19.713406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:19.732403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684c7f5-cbd6-4856-b651-abe7fb429734 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:19.749812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.750670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.751092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.751525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.752391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] RBAC: Adding query filter params (group.id=ec73010a7d3344249232dd06d49f2b9f, role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=5a9037a124a2489992eba863bf5e9b44) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:19.752856 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.754457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.762760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 184/377] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:19 2022] GET /identity/v3/role_assignments?group.id=ec73010a7d3344249232dd06d49f2b9f&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=eb7bedf01e0744eeb26681978db4121c => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:19.774864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d011afc-d157-4ab6-9b2c-1a54e11d051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.776238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d011afc-d157-4ab6-9b2c-1a54e11d051c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.776636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d011afc-d157-4ab6-9b2c-1a54e11d051c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.777011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d011afc-d157-4ab6-9b2c-1a54e11d051c None admin] PATH_INFO: `/v3/projects/af9d764870ec46c58ab37a817dc46423` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.778418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d011afc-d157-4ab6-9b2c-1a54e11d051c None admin] RBAC: Authorizing `identity:get_project(project_id=af9d764870ec46c58ab37a817dc46423)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.780056 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d011afc-d157-4ab6-9b2c-1a54e11d051c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.782167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 194/378] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:19 2022] GET /identity/v3/projects/af9d764870ec46c58ab37a817dc46423 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:19.787704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:19.923061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8fc3e-1476-4859-8b66-af8e3bb1e7a8 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:19.973846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e7ca6-fcf6-420d-9dec-eca10e70e87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:19.974516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7ca6-fcf6-420d-9dec-eca10e70e87d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:19.974803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7ca6-fcf6-420d-9dec-eca10e70e87d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:19.975078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e7ca6-fcf6-420d-9dec-eca10e70e87d None admin] PATH_INFO: `/v3/groups/d8512daeea9d4a81959239299a5d6bdd` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:19.981581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e7ca6-fcf6-420d-9dec-eca10e70e87d None admin] RBAC: Authorizing `identity:get_group(group_id=d8512daeea9d4a81959239299a5d6bdd)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:19.983063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e7ca6-fcf6-420d-9dec-eca10e70e87d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:19.984990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 185/379] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:19 2022] GET /identity/v3/groups/d8512daeea9d4a81959239299a5d6bdd => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:19.990881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d011afc-d157-4ab6-9b2c-1a54e11d051c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:20.124590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d011afc-d157-4ab6-9b2c-1a54e11d051c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:20.142823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:20.163997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b2a8f-014a-4b3a-98cf-9b50849f9a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:20.164516 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b2a8f-014a-4b3a-98cf-9b50849f9a04 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:20.164786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b2a8f-014a-4b3a-98cf-9b50849f9a04 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:20.165051 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b2a8f-014a-4b3a-98cf-9b50849f9a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:20.166387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b2a8f-014a-4b3a-98cf-9b50849f9a04 None admin] RBAC: Adding query filter params (group.id=d8512daeea9d4a81959239299a5d6bdd, role.id=b131b61299154544a627315649090a49, scope.project.id=af9d764870ec46c58ab37a817dc46423) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:20.166694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b2a8f-014a-4b3a-98cf-9b50849f9a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:20.168120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b2a8f-014a-4b3a-98cf-9b50849f9a04 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:20.175287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 195/380] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:19 2022] GET /identity/v3/role_assignments?group.id=d8512daeea9d4a81959239299a5d6bdd&scope.project.id=af9d764870ec46c58ab37a817dc46423&role.id=b131b61299154544a627315649090a49 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:20.758353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd0de0-1972-43cc-a4f2-f77546aa6074 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:20.758870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd0de0-1972-43cc-a4f2-f77546aa6074 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:20.758870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd0de0-1972-43cc-a4f2-f77546aa6074 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:20.761121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 186/381] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:20.775372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ec825-8cfb-441e-bb45-b2de0db5c09c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:20.775802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ec825-8cfb-441e-bb45-b2de0db5c09c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:20.775978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ec825-8cfb-441e-bb45-b2de0db5c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:20.799297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-158ec825-8cfb-441e-bb45-b2de0db5c09c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:20.800481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-158ec825-8cfb-441e-bb45-b2de0db5c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:20.868464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 196/382] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:21.319500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd0de0-1972-43cc-a4f2-f77546aa6074 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:21.358570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:21.445698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd0de0-1972-43cc-a4f2-f77546aa6074 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:21.495775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d85fdf-4578-49c8-8e97-35a7e63f7607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:21.496422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d85fdf-4578-49c8-8e97-35a7e63f7607 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:21.496700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d85fdf-4578-49c8-8e97-35a7e63f7607 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:21.496972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d85fdf-4578-49c8-8e97-35a7e63f7607 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:21.507942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d85fdf-4578-49c8-8e97-35a7e63f7607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-94d85fdf-4578-49c8-8e97-35a7e63f7607 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:21.514032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:21.514959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:21.515721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 187/383] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:21.521879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ec825-8cfb-441e-bb45-b2de0db5c09c None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:21.654877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07276f-1937-474d-be2a-ed45d0ff9597 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:21.655168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07276f-1937-474d-be2a-ed45d0ff9597 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:21.655354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07276f-1937-474d-be2a-ed45d0ff9597 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:21.656675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 188/384] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:35:21.656849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ec825-8cfb-441e-bb45-b2de0db5c09c None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:21.666645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986d97a-4338-4d10-8744-62882f126724 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:21.666928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986d97a-4338-4d10-8744-62882f126724 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:21.667201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986d97a-4338-4d10-8744-62882f126724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:21.676137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:21.693837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7986d97a-4338-4d10-8744-62882f126724 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:21.695032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7986d97a-4338-4d10-8744-62882f126724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:21.722543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:21.723161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:21.724374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:21.724659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:21.725347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:21.725650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:21.727029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:21.748212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 197/385] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:21.754530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:21.839871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 189/386] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:21.908926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1520a680-d3a7-4f23-b0c3-9c918328479b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:21.969303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754c338-f187-4f8b-8ef7-411d75e7f8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:21.970300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754c338-f187-4f8b-8ef7-411d75e7f8ea None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:21.970782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754c338-f187-4f8b-8ef7-411d75e7f8ea None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:21.971255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754c338-f187-4f8b-8ef7-411d75e7f8ea None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:21.978190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754c338-f187-4f8b-8ef7-411d75e7f8ea None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:21.980359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754c338-f187-4f8b-8ef7-411d75e7f8ea None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:21.982190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 198/387] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:21 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:21.987850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986d97a-4338-4d10-8744-62882f126724 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:22.027031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:22.129216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986d97a-4338-4d10-8744-62882f126724 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:22.170251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881ddc0-bf84-48d9-bed4-d0424aae57ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:22.170807 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ddc0-bf84-48d9-bed4-d0424aae57ff None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.171080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ddc0-bf84-48d9-bed4-d0424aae57ff None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.171350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ddc0-bf84-48d9-bed4-d0424aae57ff None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.173732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ddc0-bf84-48d9-bed4-d0424aae57ff None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:22.175150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ddc0-bf84-48d9-bed4-d0424aae57ff None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:22.176825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 190/388] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:21 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:22.182957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754c338-f187-4f8b-8ef7-411d75e7f8ea None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:22.207585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b916a81-bed2-4f8c-84e9-3cc517583c12 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.208213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b916a81-bed2-4f8c-84e9-3cc517583c12 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.208704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b916a81-bed2-4f8c-84e9-3cc517583c12 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.210561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 191/389] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:22.215747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd0099-dcb7-4d45-b079-661c9de89689 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.216295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd0099-dcb7-4d45-b079-661c9de89689 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.216769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd0099-dcb7-4d45-b079-661c9de89689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.236202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1bdd0099-dcb7-4d45-b079-661c9de89689 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:22.237764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1bdd0099-dcb7-4d45-b079-661c9de89689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:22.253117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:22.320314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754c338-f187-4f8b-8ef7-411d75e7f8ea None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:22.335489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 192/390] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:22.336427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd0099-dcb7-4d45-b079-661c9de89689 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:22.368280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:22.375143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4f1e2-2d22-4df1-af8a-c8e999aa612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:22.375813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4f1e2-2d22-4df1-af8a-c8e999aa612f None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.376088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4f1e2-2d22-4df1-af8a-c8e999aa612f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.376361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4f1e2-2d22-4df1-af8a-c8e999aa612f None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9/groups/ec73010a7d3344249232dd06d49f2b9f/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.380856 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4f1e2-2d22-4df1-af8a-c8e999aa612f None admin] RBAC: Authorizing `identity:create_grant(project_id=b6271cacfed747da992ff32e5f4813f9, group_id=ec73010a7d3344249232dd06d49f2b9f, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:22.382335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4f1e2-2d22-4df1-af8a-c8e999aa612f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:22.408136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 199/391] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 01:35:22 2022] PUT /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9/groups/ec73010a7d3344249232dd06d49f2b9f/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:22.502321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd0099-dcb7-4d45-b079-661c9de89689 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:22.540529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154772c-36b1-4fda-ad4b-15cc230fea79 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.541319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154772c-36b1-4fda-ad4b-15cc230fea79 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.541809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154772c-36b1-4fda-ad4b-15cc230fea79 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.546141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ad6dc-6fe5-43af-a99a-1368ac7fb885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:22.546583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 200/392] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:35:22.546790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad6dc-6fe5-43af-a99a-1368ac7fb885 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.548219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad6dc-6fe5-43af-a99a-1368ac7fb885 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.548219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad6dc-6fe5-43af-a99a-1368ac7fb885 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.550230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5401ce3-e03c-4a0c-b92e-0157761b12fe None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.550736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5401ce3-e03c-4a0c-b92e-0157761b12fe None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.551257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5401ce3-e03c-4a0c-b92e-0157761b12fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.563859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ad6dc-6fe5-43af-a99a-1368ac7fb885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-533ad6dc-6fe5-43af-a99a-1368ac7fb885 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:22.576438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:22.577844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 193/393] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:35:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:22.578621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ad6dc-6fe5-43af-a99a-1368ac7fb885 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:22.589688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a5401ce3-e03c-4a0c-b92e-0157761b12fe None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:22.591473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a5401ce3-e03c-4a0c-b92e-0157761b12fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:22.706686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 201/394] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:22.707296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:22.708596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5401ce3-e03c-4a0c-b92e-0157761b12fe None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:22.731893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ad6dc-6fe5-43af-a99a-1368ac7fb885 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:22.746967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:22.778016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01a54d-cf70-45c9-8ceb-326ab73219bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:22.778732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a54d-cf70-45c9-8ceb-326ab73219bb None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.779084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a54d-cf70-45c9-8ceb-326ab73219bb None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.779354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01a54d-cf70-45c9-8ceb-326ab73219bb None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.780017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01a54d-cf70-45c9-8ceb-326ab73219bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:22.780318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01a54d-cf70-45c9-8ceb-326ab73219bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:22.782014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01a54d-cf70-45c9-8ceb-326ab73219bb None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:22.791196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 194/395] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:22.792543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f35ce-f2a7-45b6-ba89-c897eb314c63 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.792826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f35ce-f2a7-45b6-ba89-c897eb314c63 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.793119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f35ce-f2a7-45b6-ba89-c897eb314c63 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.794619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 195/396] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:22.797445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f35ce-f2a7-45b6-ba89-c897eb314c63 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:22.843637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5401ce3-e03c-4a0c-b92e-0157761b12fe None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:22.888525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154f397-7ca5-4929-bb06-014d441446bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:22.889449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f397-7ca5-4929-bb06-014d441446bb None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.889733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f397-7ca5-4929-bb06-014d441446bb None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.890006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154f397-7ca5-4929-bb06-014d441446bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.910826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f35ce-f2a7-45b6-ba89-c897eb314c63 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:22.911333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154f397-7ca5-4929-bb06-014d441446bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-2154f397-7ca5-4929-bb06-014d441446bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:22.924929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:22.926020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:22.926794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 202/397] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:22.932047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5becb9-717a-4374-875e-8b0508b5f7f7 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.932047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5becb9-717a-4374-875e-8b0508b5f7f7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.932047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5becb9-717a-4374-875e-8b0508b5f7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.950550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f58ff-fdf5-477a-966b-5f0d277a5be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:22.951245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f58ff-fdf5-477a-966b-5f0d277a5be5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:22.951467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4b5becb9-717a-4374-875e-8b0508b5f7f7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:22.951570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f58ff-fdf5-477a-966b-5f0d277a5be5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:22.951782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f58ff-fdf5-477a-966b-5f0d277a5be5 None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:22.952676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4b5becb9-717a-4374-875e-8b0508b5f7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:22.952917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f58ff-fdf5-477a-966b-5f0d277a5be5 None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:22.954430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f58ff-fdf5-477a-966b-5f0d277a5be5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:22.956294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 196/398] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:22 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:22.957157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f58ff-fdf5-477a-966b-5f0d277a5be5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:22.981045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:23.045572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 203/399] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:23.046358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5becb9-717a-4374-875e-8b0508b5f7f7 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.073104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:23.087759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f58ff-fdf5-477a-966b-5f0d277a5be5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.126117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.127072 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.127553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.128047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.129033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] RBAC: Adding query filter params (name=member) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:23.129593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:23.131520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:23.141075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 197/400] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:23.142261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.173404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5becb9-717a-4374-875e-8b0508b5f7f7 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.219000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940ea92-4e4d-4dee-be2a-2468867d5b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.219724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940ea92-4e4d-4dee-be2a-2468867d5b11 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.220077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940ea92-4e4d-4dee-be2a-2468867d5b11 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.220561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940ea92-4e4d-4dee-be2a-2468867d5b11 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.222712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940ea92-4e4d-4dee-be2a-2468867d5b11 None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:23.225024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940ea92-4e4d-4dee-be2a-2468867d5b11 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:23.226526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:23.233203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 204/401] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:23.234796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940ea92-4e4d-4dee-be2a-2468867d5b11 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.262652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd5d91-f03e-4405-89c4-09819e08fa27 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.298581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca5228-043e-438b-8fbf-803e63c879c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.299541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca5228-043e-438b-8fbf-803e63c879c1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.299998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca5228-043e-438b-8fbf-803e63c879c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.300513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca5228-043e-438b-8fbf-803e63c879c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.311721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cca5228-043e-438b-8fbf-803e63c879c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-4cca5228-043e-438b-8fbf-803e63c879c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:23.317629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:23.318521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:23.319315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:23.321707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 198/402] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:23.322813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca5228-043e-438b-8fbf-803e63c879c1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.348257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:23.365434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940ea92-4e4d-4dee-be2a-2468867d5b11 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.400667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d06e0-02de-4822-8ca3-d83ee91ccff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.401333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d06e0-02de-4822-8ca3-d83ee91ccff0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.401613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d06e0-02de-4822-8ca3-d83ee91ccff0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.401880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d06e0-02de-4822-8ca3-d83ee91ccff0 None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.402998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d06e0-02de-4822-8ca3-d83ee91ccff0 None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:23.404435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d06e0-02de-4822-8ca3-d83ee91ccff0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:23.406042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 205/403] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:23.406800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d06e0-02de-4822-8ca3-d83ee91ccff0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.437402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca5228-043e-438b-8fbf-803e63c879c1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.471982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:23.479374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b83771-09d8-466b-ba72-9929913d6797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.480040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b83771-09d8-466b-ba72-9929913d6797 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.480322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b83771-09d8-466b-ba72-9929913d6797 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.480599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b83771-09d8-466b-ba72-9929913d6797 None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9/groups/ec73010a7d3344249232dd06d49f2b9f/roles/eb7bedf01e0744eeb26681978db4121c` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.482803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b83771-09d8-466b-ba72-9929913d6797 None admin] RBAC: Authorizing `identity:create_grant(project_id=b6271cacfed747da992ff32e5f4813f9, group_id=ec73010a7d3344249232dd06d49f2b9f, role_id=eb7bedf01e0744eeb26681978db4121c)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:23.485326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b83771-09d8-466b-ba72-9929913d6797 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:23.500288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 199/404] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 01:35:23 2022] PUT /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9/groups/ec73010a7d3344249232dd06d49f2b9f/roles/eb7bedf01e0744eeb26681978db4121c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:23.501539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b83771-09d8-466b-ba72-9929913d6797 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.523813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d06e0-02de-4822-8ca3-d83ee91ccff0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.564995 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.565704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.566038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.566385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.567143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:23.567523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:23.569029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:23.578561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 206/405] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:23.579532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.600139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:23.635107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b83771-09d8-466b-ba72-9929913d6797 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.675175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58a207-3b09-485e-9ce3-ce7b054a3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.676144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58a207-3b09-485e-9ce3-ce7b054a3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.676665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58a207-3b09-485e-9ce3-ce7b054a3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.677295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58a207-3b09-485e-9ce3-ce7b054a3ca3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.690392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d310d92-8500-4ad5-a432-53db700db195 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.691423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58a207-3b09-485e-9ce3-ce7b054a3ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-bf58a207-3b09-485e-9ce3-ce7b054a3ca3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:23.698018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:23.699039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:23.699804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:23.702237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 200/406] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:23.703209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58a207-3b09-485e-9ce3-ce7b054a3ca3 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.732647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24bd6e-75f6-4220-a140-3634e41a6f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.732944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24bd6e-75f6-4220-a140-3634e41a6f0c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.733453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24bd6e-75f6-4220-a140-3634e41a6f0c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.733822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24bd6e-75f6-4220-a140-3634e41a6f0c None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.736596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24bd6e-75f6-4220-a140-3634e41a6f0c None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:23.738836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24bd6e-75f6-4220-a140-3634e41a6f0c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:23.741249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 207/407] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:23.743327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24bd6e-75f6-4220-a140-3634e41a6f0c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.765855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:23.832299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58a207-3b09-485e-9ce3-ce7b054a3ca3 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.834709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:23.852165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24bd6e-75f6-4220-a140-3634e41a6f0c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:23.871620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fb9af-deb9-4d65-99ff-a8b99186f8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.872576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fb9af-deb9-4d65-99ff-a8b99186f8a2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.873046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fb9af-deb9-4d65-99ff-a8b99186f8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.873537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fb9af-deb9-4d65-99ff-a8b99186f8a2 None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.874954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fb9af-deb9-4d65-99ff-a8b99186f8a2 None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:23.876573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fb9af-deb9-4d65-99ff-a8b99186f8a2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:23.878986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 201/408] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:23.880009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fb9af-deb9-4d65-99ff-a8b99186f8a2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.900954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74091b12-c0dd-405d-a887-71316898432a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:23.901837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74091b12-c0dd-405d-a887-71316898432a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:23.902309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74091b12-c0dd-405d-a887-71316898432a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:23.902792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74091b12-c0dd-405d-a887-71316898432a None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:23.903762 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74091b12-c0dd-405d-a887-71316898432a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:23.904258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74091b12-c0dd-405d-a887-71316898432a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:23.905888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74091b12-c0dd-405d-a887-71316898432a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:23.923011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 208/409] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:23.924907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74091b12-c0dd-405d-a887-71316898432a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:23.997085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fb9af-deb9-4d65-99ff-a8b99186f8a2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:24.012977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:24.030408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976e5d7-fbf0-4a15-9c8d-2e3e8dc30748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:24.031326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976e5d7-fbf0-4a15-9c8d-2e3e8dc30748 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:24.031722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976e5d7-fbf0-4a15-9c8d-2e3e8dc30748 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:24.032095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976e5d7-fbf0-4a15-9c8d-2e3e8dc30748 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44/groups/ec73010a7d3344249232dd06d49f2b9f/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:24.036609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976e5d7-fbf0-4a15-9c8d-2e3e8dc30748 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a9037a124a2489992eba863bf5e9b44, group_id=ec73010a7d3344249232dd06d49f2b9f, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:24.038369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e976e5d7-fbf0-4a15-9c8d-2e3e8dc30748 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:24.055555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74091b12-c0dd-405d-a887-71316898432a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:24.056796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 202/410] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 01:35:23 2022] PUT /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44/groups/ec73010a7d3344249232dd06d49f2b9f/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:24.058162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976e5d7-fbf0-4a15-9c8d-2e3e8dc30748 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:24.106279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19668232-943a-42af-bd95-101b6ec59427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:24.106842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19668232-943a-42af-bd95-101b6ec59427 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:24.107121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19668232-943a-42af-bd95-101b6ec59427 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:24.107396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19668232-943a-42af-bd95-101b6ec59427 None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:24.108618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19668232-943a-42af-bd95-101b6ec59427 None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:24.110908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19668232-943a-42af-bd95-101b6ec59427 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:24.113846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 209/411] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:23 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:24.119855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19668232-943a-42af-bd95-101b6ec59427 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:24.142373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:24.249997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976e5d7-fbf0-4a15-9c8d-2e3e8dc30748 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:24.312634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19668232-943a-42af-bd95-101b6ec59427 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:24.317969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018218a-e5f0-4bc9-a970-1046f60cb147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:24.318859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018218a-e5f0-4bc9-a970-1046f60cb147 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:24.319362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018218a-e5f0-4bc9-a970-1046f60cb147 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:24.320362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018218a-e5f0-4bc9-a970-1046f60cb147 None admin] PATH_INFO: `/v3/groups/d8512daeea9d4a81959239299a5d6bdd` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:24.321864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018218a-e5f0-4bc9-a970-1046f60cb147 None admin] RBAC: Authorizing `identity:get_group(group_id=d8512daeea9d4a81959239299a5d6bdd)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:24.324823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018218a-e5f0-4bc9-a970-1046f60cb147 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:24.327546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 203/412] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:24 2022] GET /identity/v3/groups/d8512daeea9d4a81959239299a5d6bdd => generated 229 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:24.334419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018218a-e5f0-4bc9-a970-1046f60cb147 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:24.355483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2f2e-588a-43a1-b1c1-81501a0f03ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:24.356154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2f2e-588a-43a1-b1c1-81501a0f03ba None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:24.356492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2f2e-588a-43a1-b1c1-81501a0f03ba None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:24.356704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2f2e-588a-43a1-b1c1-81501a0f03ba None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44/groups/ec73010a7d3344249232dd06d49f2b9f/roles/eb7bedf01e0744eeb26681978db4121c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:24.358901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d2f2e-588a-43a1-b1c1-81501a0f03ba None admin] RBAC: Authorizing `identity:create_grant(project_id=5a9037a124a2489992eba863bf5e9b44, group_id=ec73010a7d3344249232dd06d49f2b9f, role_id=eb7bedf01e0744eeb26681978db4121c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:24.360455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d2f2e-588a-43a1-b1c1-81501a0f03ba None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:24.379826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 210/413] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 01:35:24 2022] PUT /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44/groups/ec73010a7d3344249232dd06d49f2b9f/roles/eb7bedf01e0744eeb26681978db4121c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:35:24.394964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:24.467816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018218a-e5f0-4bc9-a970-1046f60cb147 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:24.516840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05ea66-7229-4fed-a95d-0bf5361bbbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:24.517500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05ea66-7229-4fed-a95d-0bf5361bbbaa None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:24.517776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05ea66-7229-4fed-a95d-0bf5361bbbaa None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:24.518044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05ea66-7229-4fed-a95d-0bf5361bbbaa None admin] PATH_INFO: `/v3/projects/af9d764870ec46c58ab37a817dc46423` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:24.519152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05ea66-7229-4fed-a95d-0bf5361bbbaa None admin] RBAC: Authorizing `identity:get_project(project_id=af9d764870ec46c58ab37a817dc46423)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:24.520596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05ea66-7229-4fed-a95d-0bf5361bbbaa None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:24.527135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 204/414] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:24 2022] GET /identity/v3/projects/af9d764870ec46c58ab37a817dc46423 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:24.532733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2f2e-588a-43a1-b1c1-81501a0f03ba None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:24.648008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:24.676703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2f2e-588a-43a1-b1c1-81501a0f03ba None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:24.714024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f12256-4b12-4e6a-a997-88d6aea51c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:24.714681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12256-4b12-4e6a-a997-88d6aea51c58 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:24.714957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12256-4b12-4e6a-a997-88d6aea51c58 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:24.715226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12256-4b12-4e6a-a997-88d6aea51c58 None admin] PATH_INFO: `/v3/projects/af9d764870ec46c58ab37a817dc46423/groups/d8512daeea9d4a81959239299a5d6bdd/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:24.717208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f12256-4b12-4e6a-a997-88d6aea51c58 None admin] RBAC: Authorizing `identity:create_grant(project_id=af9d764870ec46c58ab37a817dc46423, group_id=d8512daeea9d4a81959239299a5d6bdd, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:24.718678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f12256-4b12-4e6a-a997-88d6aea51c58 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:24.735646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 211/415] 10.222.0.15 () {58 vars in 1442 bytes} [Fri Sep 9 01:35:24 2022] PUT /identity/v3/projects/af9d764870ec46c58ab37a817dc46423/groups/d8512daeea9d4a81959239299a5d6bdd/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:24.951262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19648f51-e146-4536-9e60-0da788a504bc None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:24.951556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19648f51-e146-4536-9e60-0da788a504bc None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:24.951808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19648f51-e146-4536-9e60-0da788a504bc None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:24.953320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 205/416] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:24.958821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4d1a-04d7-4b02-9edb-ad883c0e0238 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:24.959276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4d1a-04d7-4b02-9edb-ad883c0e0238 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:24.959365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a4d1a-04d7-4b02-9edb-ad883c0e0238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:24.979288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7d2a4d1a-04d7-4b02-9edb-ad883c0e0238 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:24.980464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7d2a4d1a-04d7-4b02-9edb-ad883c0e0238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:25.054322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 212/417] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:25.330029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19648f51-e146-4536-9e60-0da788a504bc None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:25.437118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19648f51-e146-4536-9e60-0da788a504bc None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:25.454662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:25.473246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3c060-b873-40d3-b060-05a858251a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:25.473886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3c060-b873-40d3-b060-05a858251a91 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:25.474165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3c060-b873-40d3-b060-05a858251a91 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:25.474438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3c060-b873-40d3-b060-05a858251a91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:25.485625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3c060-b873-40d3-b060-05a858251a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-a5a3c060-b873-40d3-b060-05a858251a91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:25.490945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:25.491831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:25.492600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 206/418] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:25.498239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a4d1a-04d7-4b02-9edb-ad883c0e0238 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:25.595313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:25.646792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a4d1a-04d7-4b02-9edb-ad883c0e0238 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:25.697049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:25.697688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:25.697971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:25.698242 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:25.698915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:25.699220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:25.700595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:25.709703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 213/419] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:25.715845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3c060-b873-40d3-b060-05a858251a91 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:25.832302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3c060-b873-40d3-b060-05a858251a91 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:25.866616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75380aa9-943a-4efc-bab2-52f91e5bd42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:25.867147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75380aa9-943a-4efc-bab2-52f91e5bd42e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:25.867425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75380aa9-943a-4efc-bab2-52f91e5bd42e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:25.867696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75380aa9-943a-4efc-bab2-52f91e5bd42e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:25.914445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:25.943103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75380aa9-943a-4efc-bab2-52f91e5bd42e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:25.944567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75380aa9-943a-4efc-bab2-52f91e5bd42e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:26.048812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 207/420] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:26.058518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:26.164292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cec7a-0aff-4399-9884-f6ee74b7455e None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:26.164477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cec7a-0aff-4399-9884-f6ee74b7455e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:26.164754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cec7a-0aff-4399-9884-f6ee74b7455e None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:26.166919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 208/421] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:26.171696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93258de-af73-4ecb-921e-2360b78a018d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:26.171978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93258de-af73-4ecb-921e-2360b78a018d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:26.172252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93258de-af73-4ecb-921e-2360b78a018d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:26.193043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d93258de-af73-4ecb-921e-2360b78a018d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:26.194268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d93258de-af73-4ecb-921e-2360b78a018d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:26.247274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20542811-74c7-420f-9705-f51069db72f8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:26.279381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:26.287058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 209/422] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:26.296185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0ccf7-6c7b-4fa9-95b3-0f03332482eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:26.296875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ccf7-6c7b-4fa9-95b3-0f03332482eb None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:26.297192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ccf7-6c7b-4fa9-95b3-0f03332482eb None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:26.297471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0ccf7-6c7b-4fa9-95b3-0f03332482eb None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:26.298618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0ccf7-6c7b-4fa9-95b3-0f03332482eb None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:26.300045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0ccf7-6c7b-4fa9-95b3-0f03332482eb None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:26.301938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 214/423] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:26 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:26.307977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93258de-af73-4ecb-921e-2360b78a018d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:26.316721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:26.446567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93258de-af73-4ecb-921e-2360b78a018d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:26.492193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4be82f-5167-4658-9817-868e92d3d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:26.492836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4be82f-5167-4658-9817-868e92d3d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:26.493140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4be82f-5167-4658-9817-868e92d3d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:26.493416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4be82f-5167-4658-9817-868e92d3d1d4 None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:26.494654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4be82f-5167-4658-9817-868e92d3d1d4 None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:26.496189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4be82f-5167-4658-9817-868e92d3d1d4 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:26.497802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 210/424] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:26 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:26.503708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0ccf7-6c7b-4fa9-95b3-0f03332482eb None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:26.637044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0ccf7-6c7b-4fa9-95b3-0f03332482eb None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:26.683702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab86c-706f-42b2-afb4-588180fddac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:26.684318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab86c-706f-42b2-afb4-588180fddac8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:26.684597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab86c-706f-42b2-afb4-588180fddac8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:26.684873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab86c-706f-42b2-afb4-588180fddac8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:26.685635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab86c-706f-42b2-afb4-588180fddac8 None admin] RBAC: Adding query filter params (group.id=ec73010a7d3344249232dd06d49f2b9f, role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=b6271cacfed747da992ff32e5f4813f9) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:26.685945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab86c-706f-42b2-afb4-588180fddac8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:26.687366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab86c-706f-42b2-afb4-588180fddac8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:26.694980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 215/425] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:26 2022] GET /identity/v3/role_assignments?group.id=ec73010a7d3344249232dd06d49f2b9f&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=18bebd2151824cdfa386be482ff8114b => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:26.844912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4be82f-5167-4658-9817-868e92d3d1d4 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:26.850682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a839b-d84a-4232-8f54-eacbf517fab4 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:26.850997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a839b-d84a-4232-8f54-eacbf517fab4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:26.851272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a839b-d84a-4232-8f54-eacbf517fab4 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:26.852863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 216/426] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:26.868057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eb6d0-9ef3-4847-a051-d00b4ac37be4 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:26.868350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eb6d0-9ef3-4847-a051-d00b4ac37be4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:26.868701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8eb6d0-9ef3-4847-a051-d00b4ac37be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:26.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-cd8eb6d0-9ef3-4847-a051-d00b4ac37be4 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:26.899779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cd8eb6d0-9ef3-4847-a051-d00b4ac37be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:26.962648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:27.004502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:27.009875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4be82f-5167-4658-9817-868e92d3d1d4 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:27.032059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 217/427] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:27.063557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4712f39-ed34-463c-812f-8ff80e2c312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:27.064474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4712f39-ed34-463c-812f-8ff80e2c312f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.064898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4712f39-ed34-463c-812f-8ff80e2c312f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.065346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4712f39-ed34-463c-812f-8ff80e2c312f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.077729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4712f39-ed34-463c-812f-8ff80e2c312f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-c4712f39-ed34-463c-812f-8ff80e2c312f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:27.083261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:27.084484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:27.085527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:27.088592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 211/428] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:35:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:27.094655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8eb6d0-9ef3-4847-a051-d00b4ac37be4 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:27.200410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c56c8c-c9ed-448b-be41-1aad33f30629 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.201026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c56c8c-c9ed-448b-be41-1aad33f30629 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.201534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c56c8c-c9ed-448b-be41-1aad33f30629 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.203206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 212/429] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:27.207625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8eb6d0-9ef3-4847-a051-d00b4ac37be4 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:27.215073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eb085-7e7f-484a-a0be-0698e6743983 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.215432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eb085-7e7f-484a-a0be-0698e6743983 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.215781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3eb085-7e7f-484a-a0be-0698e6743983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.238515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9c3eb085-7e7f-484a-a0be-0698e6743983 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:27.239868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9c3eb085-7e7f-484a-a0be-0698e6743983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:27.248428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:27.249413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.249862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.250289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.251123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:27.251592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:27.253748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:27.265930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 218/430] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:27.273306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:27.336698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:27.337611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 213/431] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:27.362215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eb085-7e7f-484a-a0be-0698e6743983 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:27.398280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545cb01-de25-4a3d-b105-017092e60350 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:27.442095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d09f-537a-4790-b51b-c89a5eb771cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:27.443152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d09f-537a-4790-b51b-c89a5eb771cf None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.443741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d09f-537a-4790-b51b-c89a5eb771cf None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.444259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d09f-537a-4790-b51b-c89a5eb771cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.449431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:27.491766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3eb085-7e7f-484a-a0be-0698e6743983 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:27.531888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ea8e8-8388-4453-a303-38619f89bfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:27.532667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ea8e8-8388-4453-a303-38619f89bfcc None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.533007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ea8e8-8388-4453-a303-38619f89bfcc None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.533377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ea8e8-8388-4453-a303-38619f89bfcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.533638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d09f-537a-4790-b51b-c89a5eb771cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:27.535311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d09f-537a-4790-b51b-c89a5eb771cf None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:27.547011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ea8e8-8388-4453-a303-38619f89bfcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e35ea8e8-8388-4453-a303-38619f89bfcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:27.553272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:27.554241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:27.555019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 214/432] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:27.563657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ea8e8-8388-4453-a303-38619f89bfcc None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:27.612887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:27.698377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 219/433] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:27.699376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d09f-537a-4790-b51b-c89a5eb771cf None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:27.710026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ea8e8-8388-4453-a303-38619f89bfcc None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:27.733942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:27.745291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37aafc-e00e-4496-aa80-a4a880f58358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:27.745859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37aafc-e00e-4496-aa80-a4a880f58358 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.746137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37aafc-e00e-4496-aa80-a4a880f58358 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.746411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37aafc-e00e-4496-aa80-a4a880f58358 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.747071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37aafc-e00e-4496-aa80-a4a880f58358 None admin] RBAC: Adding query filter params (name=member) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:27.747576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37aafc-e00e-4496-aa80-a4a880f58358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:27.748836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37aafc-e00e-4496-aa80-a4a880f58358 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:27.757089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 215/434] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:35:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:27.758200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8bdf8-5a75-4f83-8bee-f57f3181ee2a None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.758490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8bdf8-5a75-4f83-8bee-f57f3181ee2a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.758764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8bdf8-5a75-4f83-8bee-f57f3181ee2a None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.759986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 216/435] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:27.760673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8bdf8-5a75-4f83-8bee-f57f3181ee2a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:27.801582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d09f-537a-4790-b51b-c89a5eb771cf None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:27.833879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35024449-a825-4839-a19e-beb2d65e5394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:27.834490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35024449-a825-4839-a19e-beb2d65e5394 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.834763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35024449-a825-4839-a19e-beb2d65e5394 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.835033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35024449-a825-4839-a19e-beb2d65e5394 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.845160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35024449-a825-4839-a19e-beb2d65e5394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-35024449-a825-4839-a19e-beb2d65e5394 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:27.849861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:27.850896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:27.851704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 220/436] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:35:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:27.852342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35024449-a825-4839-a19e-beb2d65e5394 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:27.861679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb8bdf8-5a75-4f83-8bee-f57f3181ee2a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:27.863596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:27.896297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5aac6-1c38-4ba7-8972-eff344337dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:27.896980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5aac6-1c38-4ba7-8972-eff344337dbb None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.897214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5aac6-1c38-4ba7-8972-eff344337dbb None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.897484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5aac6-1c38-4ba7-8972-eff344337dbb None admin] PATH_INFO: `/v3/projects/b6271cacfed747da992ff32e5f4813f9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.898614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5aac6-1c38-4ba7-8972-eff344337dbb None admin] RBAC: Authorizing `identity:get_project(project_id=b6271cacfed747da992ff32e5f4813f9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:27.900519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5aac6-1c38-4ba7-8972-eff344337dbb None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:27.902147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 217/437] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:27 2022] GET /identity/v3/projects/b6271cacfed747da992ff32e5f4813f9 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:27.903223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9cce0-47e6-49ff-a1d8-de6e3bd3cd62 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:27.903499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9cce0-47e6-49ff-a1d8-de6e3bd3cd62 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:27.903765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9cce0-47e6-49ff-a1d8-de6e3bd3cd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:27.923527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ece9cce0-47e6-49ff-a1d8-de6e3bd3cd62 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:27.924704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ece9cce0-47e6-49ff-a1d8-de6e3bd3cd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:27.960322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35024449-a825-4839-a19e-beb2d65e5394 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:27.973634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:27.995795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 218/438] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:27.996852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9cce0-47e6-49ff-a1d8-de6e3bd3cd62 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.010595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383a39c-e831-4d8e-9e7e-0af20c050b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.011225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a39c-e831-4d8e-9e7e-0af20c050b99 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.011509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a39c-e831-4d8e-9e7e-0af20c050b99 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.011785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a39c-e831-4d8e-9e7e-0af20c050b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.083932 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383a39c-e831-4d8e-9e7e-0af20c050b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.085545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383a39c-e831-4d8e-9e7e-0af20c050b99 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.096980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:28.099892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9cce0-47e6-49ff-a1d8-de6e3bd3cd62 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.135795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.136402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.136676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.136953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.137693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:28.138000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.139393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.152843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 219/439] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:28.152843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.191392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 221/440] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:28.192529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383a39c-e831-4d8e-9e7e-0af20c050b99 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.203081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:28.262381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16148e3a-b0e5-44ac-a6ca-5280c9039a9e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.300892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301cb5-4b0e-48a7-9362-3914589f8518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.301554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301cb5-4b0e-48a7-9362-3914589f8518 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.301842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301cb5-4b0e-48a7-9362-3914589f8518 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.302111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4301cb5-4b0e-48a7-9362-3914589f8518 None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.303221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4301cb5-4b0e-48a7-9362-3914589f8518 None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.303810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383a39c-e831-4d8e-9e7e-0af20c050b99 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.304643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4301cb5-4b0e-48a7-9362-3914589f8518 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.306796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 220/441] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:28.307552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301cb5-4b0e-48a7-9362-3914589f8518 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.315973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:28.339735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed817-a8fd-4809-bcd7-a226656d81d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.340327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed817-a8fd-4809-bcd7-a226656d81d9 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.340609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed817-a8fd-4809-bcd7-a226656d81d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.340877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed817-a8fd-4809-bcd7-a226656d81d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.404299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4301cb5-4b0e-48a7-9362-3914589f8518 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.406073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ed817-a8fd-4809-bcd7-a226656d81d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.407527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ed817-a8fd-4809-bcd7-a226656d81d9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.418033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:28.438408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7a1ff-62fa-45c0-ad9c-acc49025e81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.439106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7a1ff-62fa-45c0-ad9c-acc49025e81b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.439456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7a1ff-62fa-45c0-ad9c-acc49025e81b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.439821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7a1ff-62fa-45c0-ad9c-acc49025e81b None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.440942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7a1ff-62fa-45c0-ad9c-acc49025e81b None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.442409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7a1ff-62fa-45c0-ad9c-acc49025e81b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.444023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 221/442] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:28.444776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7a1ff-62fa-45c0-ad9c-acc49025e81b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.504455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 222/443] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:28.506186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed817-a8fd-4809-bcd7-a226656d81d9 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.518066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:28.548175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7a1ff-62fa-45c0-ad9c-acc49025e81b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.584483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04ac86-571e-45f3-82ba-fa41f169eb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.585138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04ac86-571e-45f3-82ba-fa41f169eb0a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.585422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04ac86-571e-45f3-82ba-fa41f169eb0a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.585693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04ac86-571e-45f3-82ba-fa41f169eb0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.595763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c04ac86-571e-45f3-82ba-fa41f169eb0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-7c04ac86-571e-45f3-82ba-fa41f169eb0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:28.601053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:28.602092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:28.602900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 222/444] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:28.603569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04ac86-571e-45f3-82ba-fa41f169eb0a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.610534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed817-a8fd-4809-bcd7-a226656d81d9 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.644602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.645233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.645510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.645780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.646416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:28.646551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] RBAC: Adding query filter params (group.id=ec73010a7d3344249232dd06d49f2b9f, role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=b6271cacfed747da992ff32e5f4813f9) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:28.646828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.648666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.656288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 223/445] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/role_assignments?group.id=ec73010a7d3344249232dd06d49f2b9f&scope.project.id=b6271cacfed747da992ff32e5f4813f9&role.id=eb7bedf01e0744eeb26681978db4121c => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:28.657084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.714068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04ac86-571e-45f3-82ba-fa41f169eb0a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.742613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:28.756130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf6cd6-85cf-4245-955c-aed5d25dc239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.756776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf6cd6-85cf-4245-955c-aed5d25dc239 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.757053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf6cd6-85cf-4245-955c-aed5d25dc239 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.757351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf6cd6-85cf-4245-955c-aed5d25dc239 None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.758513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf6cd6-85cf-4245-955c-aed5d25dc239 None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.760518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf6cd6-85cf-4245-955c-aed5d25dc239 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.767758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 223/446] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:28.767758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf6cd6-85cf-4245-955c-aed5d25dc239 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.775893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf085d-3ae2-4f54-af13-27d5cfc8701d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.813374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73861378-709f-41cf-a2b2-b3e2feb9c25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.814014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73861378-709f-41cf-a2b2-b3e2feb9c25a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.814292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73861378-709f-41cf-a2b2-b3e2feb9c25a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.814728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73861378-709f-41cf-a2b2-b3e2feb9c25a None admin] PATH_INFO: `/v3/projects/5a9037a124a2489992eba863bf5e9b44` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.815915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73861378-709f-41cf-a2b2-b3e2feb9c25a None admin] RBAC: Authorizing `identity:get_project(project_id=5a9037a124a2489992eba863bf5e9b44)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.817401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73861378-709f-41cf-a2b2-b3e2feb9c25a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.819046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 224/447] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/projects/5a9037a124a2489992eba863bf5e9b44 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:28.819046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73861378-709f-41cf-a2b2-b3e2feb9c25a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.887465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf6cd6-85cf-4245-955c-aed5d25dc239 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.900765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:28.933105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.933308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.933538 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.933815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.935621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:28.935621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.936592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.940585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73861378-709f-41cf-a2b2-b3e2feb9c25a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:28.945415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 224/448] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:28.949107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:28.980275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:28.980643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:28.980928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:28.981235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:28.981962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] RBAC: Adding query filter params (group.id=ec73010a7d3344249232dd06d49f2b9f, role.id=18bebd2151824cdfa386be482ff8114b, scope.project.id=5a9037a124a2489992eba863bf5e9b44) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:28.982272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:28.983702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:28.992201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 225/449] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/role_assignments?group.id=ec73010a7d3344249232dd06d49f2b9f&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=18bebd2151824cdfa386be482ff8114b => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:28.992649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:29.003068 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:29.083212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f775a11-b33d-4825-b2b9-3ca41519f49a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:29.125761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd258d7-665d-4ea4-b5cb-90a89100d345 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:29.131749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f061b4d-aab2-4605-ac7e-4c3494fd2588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:29.132519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f061b4d-aab2-4605-ac7e-4c3494fd2588 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:29.132801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f061b4d-aab2-4605-ac7e-4c3494fd2588 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:29.133125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f061b4d-aab2-4605-ac7e-4c3494fd2588 None admin] PATH_INFO: `/v3/groups/ec73010a7d3344249232dd06d49f2b9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:29.134355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f061b4d-aab2-4605-ac7e-4c3494fd2588 None admin] RBAC: Authorizing `identity:get_group(group_id=ec73010a7d3344249232dd06d49f2b9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:29.135808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f061b4d-aab2-4605-ac7e-4c3494fd2588 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:29.137713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 225/450] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/groups/ec73010a7d3344249232dd06d49f2b9f => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:29.143481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f061b4d-aab2-4605-ac7e-4c3494fd2588 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:29.163225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93f738-82ac-4fe7-9494-07cc43aa421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:29.163870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93f738-82ac-4fe7-9494-07cc43aa421f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:29.164143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93f738-82ac-4fe7-9494-07cc43aa421f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:29.164410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93f738-82ac-4fe7-9494-07cc43aa421f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:29.196307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:29.244135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93f738-82ac-4fe7-9494-07cc43aa421f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:29.245632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93f738-82ac-4fe7-9494-07cc43aa421f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:29.255066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:29.273525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f061b4d-aab2-4605-ac7e-4c3494fd2588 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:29.316646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:29.317280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:29.317563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:29.318434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:29.318537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] RBAC: Adding query filter params (group.id=ec73010a7d3344249232dd06d49f2b9f, role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=5a9037a124a2489992eba863bf5e9b44) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:29.319842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:29.321347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:29.328153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 226/451] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:29 2022] GET /identity/v3/role_assignments?group.id=ec73010a7d3344249232dd06d49f2b9f&scope.project.id=5a9037a124a2489992eba863bf5e9b44&role.id=eb7bedf01e0744eeb26681978db4121c => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:29.358751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 226/452] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:29.364934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:29.473155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c886e-4164-4279-9822-2aa5138f64a8 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:29.475537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:29.508486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e29e70-bea5-41e2-a1bb-743955a18b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:29.509150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e29e70-bea5-41e2-a1bb-743955a18b07 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:29.509430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e29e70-bea5-41e2-a1bb-743955a18b07 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:29.509705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e29e70-bea5-41e2-a1bb-743955a18b07 None admin] PATH_INFO: `/v3/projects/af9d764870ec46c58ab37a817dc46423` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:29.510832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e29e70-bea5-41e2-a1bb-743955a18b07 None admin] RBAC: Authorizing `identity:get_project(project_id=af9d764870ec46c58ab37a817dc46423)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:29.512262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e29e70-bea5-41e2-a1bb-743955a18b07 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:29.514105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 227/453] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:35:29 2022] GET /identity/v3/projects/af9d764870ec46c58ab37a817dc46423 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:29.519608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93f738-82ac-4fe7-9494-07cc43aa421f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:29.621161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93f738-82ac-4fe7-9494-07cc43aa421f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:29.628989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:29.657684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefce47e-1c09-4eb1-99fe-af918fd859a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:29.658316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce47e-1c09-4eb1-99fe-af918fd859a2 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:29.658593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce47e-1c09-4eb1-99fe-af918fd859a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:29.658870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce47e-1c09-4eb1-99fe-af918fd859a2 None admin] PATH_INFO: `/v3/groups/d8512daeea9d4a81959239299a5d6bdd` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:29.659965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefce47e-1c09-4eb1-99fe-af918fd859a2 None admin] RBAC: Authorizing `identity:get_group(group_id=d8512daeea9d4a81959239299a5d6bdd)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:29.661510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefce47e-1c09-4eb1-99fe-af918fd859a2 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:29.663143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 227/454] 10.222.0.15 () {56 vars in 1180 bytes} [Fri Sep 9 01:35:29 2022] GET /identity/v3/groups/d8512daeea9d4a81959239299a5d6bdd => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:29.669287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e29e70-bea5-41e2-a1bb-743955a18b07 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:29.770385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e29e70-bea5-41e2-a1bb-743955a18b07 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:29.802116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642619f-19f8-40c8-a6ed-5a5ddede6ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:29.802706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642619f-19f8-40c8-a6ed-5a5ddede6ae5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:29.802983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642619f-19f8-40c8-a6ed-5a5ddede6ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:29.803249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642619f-19f8-40c8-a6ed-5a5ddede6ae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:29.803961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642619f-19f8-40c8-a6ed-5a5ddede6ae5 None admin] RBAC: Adding query filter params (group.id=d8512daeea9d4a81959239299a5d6bdd, role.id=b131b61299154544a627315649090a49, scope.project.id=af9d764870ec46c58ab37a817dc46423) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:29.804265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642619f-19f8-40c8-a6ed-5a5ddede6ae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:29.805708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642619f-19f8-40c8-a6ed-5a5ddede6ae5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:29.811202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 228/455] 10.222.0.15 () {56 vars in 1376 bytes} [Fri Sep 9 01:35:29 2022] GET /identity/v3/role_assignments?group.id=d8512daeea9d4a81959239299a5d6bdd&scope.project.id=af9d764870ec46c58ab37a817dc46423&role.id=b131b61299154544a627315649090a49 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:32.350543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218e5c-b714-4051-a00d-8f4b52bf79f2 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:32.351061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218e5c-b714-4051-a00d-8f4b52bf79f2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:32.351061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb218e5c-b714-4051-a00d-8f4b52bf79f2 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:32.352669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 228/456] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:32.358632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a304fc-0708-4f42-bbee-d16743c7bedf None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:32.359034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a304fc-0708-4f42-bbee-d16743c7bedf None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:32.359218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a304fc-0708-4f42-bbee-d16743c7bedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:32.381212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e4a304fc-0708-4f42-bbee-d16743c7bedf None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:32.382404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e4a304fc-0708-4f42-bbee-d16743c7bedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:32.422629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:32.475754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 229/457] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:32.532309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0deb-07c6-4b34-9d5b-45f58803a1c7 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:32.532610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0deb-07c6-4b34-9d5b-45f58803a1c7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:32.532857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0deb-07c6-4b34-9d5b-45f58803a1c7 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:32.534365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 229/458] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:32.539637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3478ae-1a9a-42ad-ba85-75377b40deed None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:32.539927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3478ae-1a9a-42ad-ba85-75377b40deed None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:32.540203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3478ae-1a9a-42ad-ba85-75377b40deed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:32.570059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1a3478ae-1a9a-42ad-ba85-75377b40deed None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:32.571245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1a3478ae-1a9a-42ad-ba85-75377b40deed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:32.664007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 230/459] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:32.804448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64779db-f319-43f6-ae15-61d1af29fc5e None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:32.804766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64779db-f319-43f6-ae15-61d1af29fc5e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:32.805003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64779db-f319-43f6-ae15-61d1af29fc5e None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:32.806697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 230/460] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:32.811990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d93541-5443-48e3-9746-bc07718691ef None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:32.812276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d93541-5443-48e3-9746-bc07718691ef None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:32.812553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d93541-5443-48e3-9746-bc07718691ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:32.836524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c7d93541-5443-48e3-9746-bc07718691ef None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:32.837905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c7d93541-5443-48e3-9746-bc07718691ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:32.851991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64779db-f319-43f6-ae15-61d1af29fc5e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:32.958403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 231/461] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:33.042145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64779db-f319-43f6-ae15-61d1af29fc5e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:33.094198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:33.094824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.095102 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.095369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.110533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:33.112066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:33.116959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d93541-5443-48e3-9746-bc07718691ef None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:33.138602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:33.139572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:33.140431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:33.140431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:33.140431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:33.142902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 231/462] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:33.151258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:33.171293 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:33.202091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:33.283471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa868ba-6725-47bd-86a7-ed7b5d457141 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:33.292875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d93541-5443-48e3-9746-bc07718691ef None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:33.317982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:33.318706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.318986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.319378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.320011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:33.320316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:33.321865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:33.326925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab79bb3-9c53-4036-8f46-497367e4d4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:33.327549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab79bb3-9c53-4036-8f46-497367e4d4a1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.327822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab79bb3-9c53-4036-8f46-497367e4d4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.328089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab79bb3-9c53-4036-8f46-497367e4d4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.334791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 232/463] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:33.338475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:33.341701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab79bb3-9c53-4036-8f46-497367e4d4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:33.343153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab79bb3-9c53-4036-8f46-497367e4d4a1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-4ab79bb3-9c53-4036-8f46-497367e4d4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:33.359452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:33.360766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:33.362093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:33.362093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:33.362093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:33.362093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 232/464] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:33.362699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f13f71-5b79-462b-92e5-bf0e23d6e3c3 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.362976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f13f71-5b79-462b-92e5-bf0e23d6e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.363246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f13f71-5b79-462b-92e5-bf0e23d6e3c3 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.364528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 233/465] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:33.365416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f13f71-5b79-462b-92e5-bf0e23d6e3c3 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:33.432133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:33.447502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:33.449501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664098f3-d86a-4a38-94ef-6d0ae9da1367 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:33.470659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f13f71-5b79-462b-92e5-bf0e23d6e3c3 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:33.486269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:33.486898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.487156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.487420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.497612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:33.499047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:33.506073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f76d1-1b31-495d-9672-244ae7d7c97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:33.506672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f76d1-1b31-495d-9672-244ae7d7c97f None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.506950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f76d1-1b31-495d-9672-244ae7d7c97f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.507224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f76d1-1b31-495d-9672-244ae7d7c97f None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.508284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f76d1-1b31-495d-9672-244ae7d7c97f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:33.510249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f76d1-1b31-495d-9672-244ae7d7c97f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:33.514280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:33.515136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:33.515956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:33.515956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:33.515956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:33.515956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 233/466] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:33.517170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:33.537176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 234/467] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:35:33 2022] POST /identity/v3/users => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:35:33.537176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3474a34-b908-493e-9c15-0ff10fdec285 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.537176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3474a34-b908-493e-9c15-0ff10fdec285 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.537485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3474a34-b908-493e-9c15-0ff10fdec285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.558119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a3474a34-b908-493e-9c15-0ff10fdec285 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:33.559397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a3474a34-b908-493e-9c15-0ff10fdec285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:33.640395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 235/468] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:33.641513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3474a34-b908-493e-9c15-0ff10fdec285 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:33.675569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beede322-0a81-493d-ba70-3364c20b7e8b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:33.722835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:33.723926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.724415 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.725310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.726549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:33.727082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:33.729712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:33.739035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:33.742839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:33.747376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 234/469] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:33.753425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:33.800277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3474a34-b908-493e-9c15-0ff10fdec285 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:33.847656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:33.848601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.849407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.849929 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.851280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:33.851770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:33.853921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:33.870394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 236/470] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:33.876109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:33.880321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a42cf-4cd6-406f-b525-c7ae541ef2b5 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:33.920175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf82b5-1347-44d6-96b9-b4d1ff32ccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:33.920612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf82b5-1347-44d6-96b9-b4d1ff32ccb1 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:33.920898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf82b5-1347-44d6-96b9-b4d1ff32ccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:33.921262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf82b5-1347-44d6-96b9-b4d1ff32ccb1 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:33.921912 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf82b5-1347-44d6-96b9-b4d1ff32ccb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:33.923350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf82b5-1347-44d6-96b9-b4d1ff32ccb1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:33.957199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 235/471] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:35:33 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:35:33.986976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf82b5-1347-44d6-96b9-b4d1ff32ccb1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:34.009810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b69c6e-1e14-4ede-955d-80f60bdfe77a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:34.039605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:34.052588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513a2c9-e832-449b-b8eb-c8705a95d8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:34.053583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513a2c9-e832-449b-b8eb-c8705a95d8d7 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:34.054045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513a2c9-e832-449b-b8eb-c8705a95d8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:34.054515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513a2c9-e832-449b-b8eb-c8705a95d8d7 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:34.056562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513a2c9-e832-449b-b8eb-c8705a95d8d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:34.059216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513a2c9-e832-449b-b8eb-c8705a95d8d7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:34.086374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:34.112315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 237/472] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:35:33 2022] POST /identity/v3/users => generated 248 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:35:34.185010 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf82b5-1347-44d6-96b9-b4d1ff32ccb1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:34.242740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00e41b-c28a-4066-a5cb-d59339026dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:34.243459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00e41b-c28a-4066-a5cb-d59339026dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:34.243745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00e41b-c28a-4066-a5cb-d59339026dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:34.244022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00e41b-c28a-4066-a5cb-d59339026dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:34.260591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00e41b-c28a-4066-a5cb-d59339026dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:34.262179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00e41b-c28a-4066-a5cb-d59339026dc5 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-3a00e41b-c28a-4066-a5cb-d59339026dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:34.279961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:34.280908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:34.281771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:34.281771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:34.281771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:34.281771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 236/473] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:34.287386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513a2c9-e832-449b-b8eb-c8705a95d8d7 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:34.323180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5bbc0-9f6a-4a77-afd4-8dd3fc68de8b None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:34.323462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5bbc0-9f6a-4a77-afd4-8dd3fc68de8b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:34.323761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb5bbc0-9f6a-4a77-afd4-8dd3fc68de8b None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:34.325130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 237/474] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:34.331737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e7a02-4fe0-477a-97e8-71231f9ecb1e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:34.333480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e7a02-4fe0-477a-97e8-71231f9ecb1e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:34.333834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e7a02-4fe0-477a-97e8-71231f9ecb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:34.356609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-440e7a02-4fe0-477a-97e8-71231f9ecb1e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:34.358155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-440e7a02-4fe0-477a-97e8-71231f9ecb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:34.422117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513a2c9-e832-449b-b8eb-c8705a95d8d7 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:34.445782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 238/475] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:34.460071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:34.460734 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:34.461007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:34.461302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:34.461942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:34.462356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:34.463723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:34.476133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 238/476] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:34.481843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e7a02-4fe0-477a-97e8-71231f9ecb1e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:34.518178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:34.599443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e7a02-4fe0-477a-97e8-71231f9ecb1e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:34.635175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4a71e-f5ed-482d-a402-4475056c96d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:34.635778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4a71e-f5ed-482d-a402-4475056c96d1 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:34.636055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4a71e-f5ed-482d-a402-4475056c96d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:34.636324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4a71e-f5ed-482d-a402-4475056c96d1 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:34.636976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4a71e-f5ed-482d-a402-4475056c96d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:34.638450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4a71e-f5ed-482d-a402-4475056c96d1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:34.664942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 239/477] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:35:34 2022] POST /identity/v3/users => generated 249 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:35:34.782319 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:34.828028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:34.913668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80db13-f93e-4366-813c-4746375b4075 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:34.969988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:34.970987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:34.971479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:34.971948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:34.972826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:34.974484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:34.990495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.sql.core [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 01:35:34.990495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 01:35:34.990495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [parameters: {'id': 'de7e71fa67d44d32991d3ad3c084200c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 01:35:34.990495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87883) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application conn.query(q) Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application result.read() Sep 09 01:35:35.008682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 01:35:35.010109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application next(self.gen) Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application yield resource Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application next(self.gen) Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise self.value Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application session.commit() Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 01:35:35.011020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self.session.flush() Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application compat.raise_( Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 01:35:35.011880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 01:35:35.012738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application conn.query(q) Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application result.read() Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 01:35:35.015073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [parameters: {'id': 'de7e71fa67d44d32991d3ad3c084200c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:35.017010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 01:35:35.019017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:35.022766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 239/478] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 01:35:34 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:35.028360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4a71e-f5ed-482d-a402-4475056c96d1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:35.141631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:35.153265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4a71e-f5ed-482d-a402-4475056c96d1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:35.189520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d978c-d3e0-485e-87cc-6eaef943d216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:35.190156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d978c-d3e0-485e-87cc-6eaef943d216 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:35.190435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d978c-d3e0-485e-87cc-6eaef943d216 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:35.190702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d978c-d3e0-485e-87cc-6eaef943d216 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:35.201250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d978c-d3e0-485e-87cc-6eaef943d216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-ea4d978c-d3e0-485e-87cc-6eaef943d216 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:35.206527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:35.207379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:35.208131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 240/479] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:35:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:35.214881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:35.338037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b74aef-1933-4cbe-8c7c-bbb46d7d8bb2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:35.342824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:35.378859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8dda8-20f2-4e84-9a15-f8fa3bcfab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:35.379500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8dda8-20f2-4e84-9a15-f8fa3bcfab43 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:35.379780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8dda8-20f2-4e84-9a15-f8fa3bcfab43 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:35.380051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8dda8-20f2-4e84-9a15-f8fa3bcfab43 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:35.380999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8dda8-20f2-4e84-9a15-f8fa3bcfab43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:35.381336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8dda8-20f2-4e84-9a15-f8fa3bcfab43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:35.382800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8dda8-20f2-4e84-9a15-f8fa3bcfab43 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:35.397233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 240/480] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:36.470028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bf1aa-aaf0-42be-a9c7-e60f5fd57fbc None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:36.470541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bf1aa-aaf0-42be-a9c7-e60f5fd57fbc None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:36.470646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bf1aa-aaf0-42be-a9c7-e60f5fd57fbc None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:36.472109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 241/481] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:36.477506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33975-d6d7-45d3-b91d-dc2082bb3cc1 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:36.478506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33975-d6d7-45d3-b91d-dc2082bb3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:36.478654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd33975-d6d7-45d3-b91d-dc2082bb3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:36.506304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-bcd33975-d6d7-45d3-b91d-dc2082bb3cc1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:36.507506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-bcd33975-d6d7-45d3-b91d-dc2082bb3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:36.579895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 241/482] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:36.892670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b89612-d9b8-497d-a45f-2a1ef1613aa3 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:36.893374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b89612-d9b8-497d-a45f-2a1ef1613aa3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:36.893855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b89612-d9b8-497d-a45f-2a1ef1613aa3 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:36.895662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 242/483] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:36.905337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e812e1-8987-45f4-bbc1-24163382bc7a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:36.905882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e812e1-8987-45f4-bbc1-24163382bc7a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:36.906342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e812e1-8987-45f4-bbc1-24163382bc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:36.933823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e0e812e1-8987-45f4-bbc1-24163382bc7a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:36.935313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e0e812e1-8987-45f4-bbc1-24163382bc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:36.966164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab19ac6-10f4-4f6f-9b37-173f18422f6e None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:36.966772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab19ac6-10f4-4f6f-9b37-173f18422f6e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:36.967245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab19ac6-10f4-4f6f-9b37-173f18422f6e None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:36.970481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 243/484] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:36.975212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4954c-56c1-4f54-8f16-a0d012f83a66 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:36.975729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4954c-56c1-4f54-8f16-a0d012f83a66 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:36.976209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4954c-56c1-4f54-8f16-a0d012f83a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.001796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f7e4954c-56c1-4f54-8f16-a0d012f83a66 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:37.004004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f7e4954c-56c1-4f54-8f16-a0d012f83a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:37.029292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 242/485] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:37.030449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e812e1-8987-45f4-bbc1-24163382bc7a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:37.082139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:37.131345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 244/486] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:37.188928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e812e1-8987-45f4-bbc1-24163382bc7a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:37.236025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c8c4f-7f0e-4207-ad01-2c26256fb236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:37.236942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c8c4f-7f0e-4207-ad01-2c26256fb236 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:37.237410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c8c4f-7f0e-4207-ad01-2c26256fb236 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:37.237850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c8c4f-7f0e-4207-ad01-2c26256fb236 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.255174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c8c4f-7f0e-4207-ad01-2c26256fb236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-f92c8c4f-7f0e-4207-ad01-2c26256fb236 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:37.261334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:37.262283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:37.263058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:37.265515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 243/487] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:37.271621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4954c-56c1-4f54-8f16-a0d012f83a66 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:37.359775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:37.380988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4954c-56c1-4f54-8f16-a0d012f83a66 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:37.392017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f436e22-253a-47d7-8fad-18261691df49 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:37.392492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f436e22-253a-47d7-8fad-18261691df49 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:37.392959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f436e22-253a-47d7-8fad-18261691df49 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.394719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 244/488] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:37.399991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25557ced-4181-46ea-897a-8c5038f67322 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:37.400476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25557ced-4181-46ea-897a-8c5038f67322 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:37.400883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25557ced-4181-46ea-897a-8c5038f67322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.421944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-25557ced-4181-46ea-897a-8c5038f67322 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:37.423717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-25557ced-4181-46ea-897a-8c5038f67322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:37.431036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:37.431659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:37.431934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:37.432203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.432869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:37.433392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:37.434764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:37.450950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 245/489] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:37.460631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:37.526694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 245/490] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:37.528122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25557ced-4181-46ea-897a-8c5038f67322 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:37.548911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:37.590108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba70da-193d-42d3-889a-2170cecb8c95 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:37.635325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:37.636992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:37.637510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:37.638254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.641892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25557ced-4181-46ea-897a-8c5038f67322 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:37.653322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:37.655670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:37.659408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:37.677837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:37.678709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:37.679536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:37.679536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:37.679536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:37.681796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 246/491] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:37.682112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bf79d-7296-486a-8e59-2dafc8ebb508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:37.682740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bf79d-7296-486a-8e59-2dafc8ebb508 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:37.683010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bf79d-7296-486a-8e59-2dafc8ebb508 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:37.683275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bf79d-7296-486a-8e59-2dafc8ebb508 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.683800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:37.693879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bf79d-7296-486a-8e59-2dafc8ebb508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-bc7bf79d-7296-486a-8e59-2dafc8ebb508 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:37.699118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:37.700043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:37.700844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:37.701545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 246/492] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:37.707091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bf79d-7296-486a-8e59-2dafc8ebb508 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:37.799131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:37.824441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bf79d-7296-486a-8e59-2dafc8ebb508 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:37.831722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1b3e0-a3cb-4259-a75e-d74bb96fadbc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:37.861154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:37.862032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:37.862479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:37.862947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.863756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:37.864221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:37.865760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9692-9548-4e71-815d-2ba6df7df95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:37.865992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:37.866379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9692-9548-4e71-815d-2ba6df7df95e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:37.866649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9692-9548-4e71-815d-2ba6df7df95e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:37.866912 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc9692-9548-4e71-815d-2ba6df7df95e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:37.878239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 247/493] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:37.878604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc9692-9548-4e71-815d-2ba6df7df95e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:37.879429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-3dfc9692-9548-4e71-815d-2ba6df7df95e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:37.884107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:37.885178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 247/494] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:37.885952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9692-9548-4e71-815d-2ba6df7df95e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:37.947587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:37.993795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc9692-9548-4e71-815d-2ba6df7df95e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.003516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7491070-6ef9-41de-b608-8be0fbea322b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.032779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f1c5-977c-4989-a042-31df2192e6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.033292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f1c5-977c-4989-a042-31df2192e6ce None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.033570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f1c5-977c-4989-a042-31df2192e6ce None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.033851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f1c5-977c-4989-a042-31df2192e6ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.044906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.045097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303f1c5-977c-4989-a042-31df2192e6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:38.045545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.045820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.046101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.046764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:38.047061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.048420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-a303f1c5-977c-4989-a042-31df2192e6ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:38.050328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:38.051155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:38.051925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 248/495] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:38.052554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f1c5-977c-4989-a042-31df2192e6ce None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.055862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:38.064513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 248/496] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:38.064513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.164600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f1c5-977c-4989-a042-31df2192e6ce None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.181564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:38.189095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f0b31-0a1b-4296-9f01-9bc6bc93f0f5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.201014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.201742 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.202015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.202286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.209796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.211239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:38.224186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:38.225418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 249/497] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:38.226308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.234058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.234676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.234955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.235224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.235884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:38.236187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.237577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.251203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 249/498] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:38.251203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.329213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:38.335610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82163e-c24c-4780-b1c9-4e4feb3ab267 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.372617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.373321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.373599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.373870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.374521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:38.374821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.376195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.377332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ac23b-047c-4ce2-bf58-a524fc712d44 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.384607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 250/499] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:38.389182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.419270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.420178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.420592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.420999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.425730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:38.436188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.437690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:38.452529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:38.453565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:38.454381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:38.454381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:38.454381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:38.454381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 250/500] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:38.454381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.489613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69571ec3-dfcb-40cc-a088-baf4c8a6a47e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.522804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.523346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.523619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.523887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.524674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:38.524978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.526449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.552030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17ec99-531c-4a76-baed-50a9b3c2de2e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.562536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 251/501] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:38.563362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.565497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:38.588000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a5e5c-4f13-442a-b537-afb6eff8bf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.588619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a5e5c-4f13-442a-b537-afb6eff8bf19 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.588905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a5e5c-4f13-442a-b537-afb6eff8bf19 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.589210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a5e5c-4f13-442a-b537-afb6eff8bf19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.599497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a5e5c-4f13-442a-b537-afb6eff8bf19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.600939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a5e5c-4f13-442a-b537-afb6eff8bf19 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-480a5e5c-4f13-442a-b537-afb6eff8bf19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:38.614717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:38.615772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:38.616726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:38.616726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:38.616726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:38.616726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 251/502] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:38.616956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae530-915e-401f-a8fa-50c6274a371d None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.617211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae530-915e-401f-a8fa-50c6274a371d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.617484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae530-915e-401f-a8fa-50c6274a371d None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.619233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 252/503] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:38.619920 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ae530-915e-401f-a8fa-50c6274a371d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.661938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:38.668870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f85fb9-e1df-4468-b11f-7aae6dae1d70 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.701848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.702460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.702734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.702996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.712519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.713981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.720102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ae530-915e-401f-a8fa-50c6274a371d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:38.728764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:38.729700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:38.730556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:38.730556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:38.730556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:38.730556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 252/504] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:38.733864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.756483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.757116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.757393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.757662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.758291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:38.758590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.759999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.771875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 253/505] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:38.772649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:38.797053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:38.832937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fbc42-6cf7-4247-99ff-95eb2199d614 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.865897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f3410-a6d8-446f-bb94-26dbe9347fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.866484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f3410-a6d8-446f-bb94-26dbe9347fca None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.866755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f3410-a6d8-446f-bb94-26dbe9347fca None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.867017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f3410-a6d8-446f-bb94-26dbe9347fca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.870637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b9a2a-56ee-4b21-93b9-fd3ed801c95e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:38.885525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:38.904606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae1c4b-ac7f-4351-a5ba-a096fddc101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:38.905229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1c4b-ac7f-4351-a5ba-a096fddc101f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.905501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1c4b-ac7f-4351-a5ba-a096fddc101f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.905764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1c4b-ac7f-4351-a5ba-a096fddc101f None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.906387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae1c4b-ac7f-4351-a5ba-a096fddc101f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:38.906682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae1c4b-ac7f-4351-a5ba-a096fddc101f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.908042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae1c4b-ac7f-4351-a5ba-a096fddc101f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:38.919379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 254/506] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:38.920553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566de90-3a56-4f2b-88db-ae717c8a863c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:38.920830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566de90-3a56-4f2b-88db-ae717c8a863c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:38.921127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566de90-3a56-4f2b-88db-ae717c8a863c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:38.938009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2566de90-3a56-4f2b-88db-ae717c8a863c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:38.939155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f3410-a6d8-446f-bb94-26dbe9347fca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:38.939271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2566de90-3a56-4f2b-88db-ae717c8a863c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:38.940616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f3410-a6d8-446f-bb94-26dbe9347fca None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.002424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 255/507] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:39.003237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566de90-3a56-4f2b-88db-ae717c8a863c None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.019258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.039307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 253/508] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:39.040411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f3410-a6d8-446f-bb94-26dbe9347fca None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.090159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.117341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566de90-3a56-4f2b-88db-ae717c8a863c None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.148594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f3410-a6d8-446f-bb94-26dbe9347fca None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.156687 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.157324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.157599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.158140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.158528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:39.158825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.160185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.176691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 256/509] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:39.176691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.189903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.190538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.190814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.191083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.198562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.199995 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:39.211459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:39.212810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 254/510] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:39.218668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.257387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.282992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd022d1-a76d-4d10-bd0c-f33fa1d92455 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.315180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac93f6-7bb6-4194-a12d-44a3e6a2e73b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.316168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.316783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.317076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.317360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.324364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.325200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.326670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:39.338940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:39.339827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:39.340743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 257/511] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:39.341490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.350574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.351203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.351477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.351748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.362066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.363525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:39.378500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:39.379397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:39.380213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:39.380213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:39.380213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:39.380213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 255/512] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:39.380672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.441328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8f868-e0c5-42ed-b4dc-26637ee93156 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.476504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.477154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.477431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.477724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.478785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663932c-1bb6-40b8-b6c1-9c8102d9c0a3 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.479611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.486696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.488327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:39.499294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:39.500334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 01:35:39.501463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:39.505041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 258/513] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:39.506178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.515757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.516358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.516636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.516911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.517661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:39.517964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.519382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.536636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 256/514] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:39.537451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.555294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.607561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8a93c-cbbd-4471-87a6-3371bc8e2a4e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.638394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1490ac-6af6-4bb1-810b-2b6b6d61f33b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.640075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.640700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.640970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.641268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.651329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.652743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:39.666925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:39.667942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:39.669329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:39.669329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:39.669329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:39.669329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 259/515] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:39.669329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.674276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.674884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.675162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.675433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.676150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:39.676453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.677944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.695162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 257/516] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:39.695943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.714149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.773297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.775914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e4e9ec-7a8b-4359-a6ab-0c7987b136b2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.799288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9011e6-3390-4924-a500-9775581b336f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.809451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.810050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.810320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.810583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.811217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:39.811513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.812854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.824767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 260/517] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:39.825603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.835192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.835787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.836056 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.836320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:39.837110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:39.837412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:39.838826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:39.855237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 258/518] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:39.856291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:39.937872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22824df6-1e5b-4cf7-8e83-b44f8acbc9fd None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.945566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:39.964624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afa06a-8aee-4393-8e93-fd5f3a0d136b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:39.976183 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17d2ec-e495-4cc8-aa77-7b89cd41d03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:39.976796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17d2ec-e495-4cc8-aa77-7b89cd41d03a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:39.977089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17d2ec-e495-4cc8-aa77-7b89cd41d03a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:39.977445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17d2ec-e495-4cc8-aa77-7b89cd41d03a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.017491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.018131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.018407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.018675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.019307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:40.019606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.020958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.023079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.034781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 259/519] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:40.035855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.056575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17d2ec-e495-4cc8-aa77-7b89cd41d03a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.058072 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17d2ec-e495-4cc8-aa77-7b89cd41d03a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.148342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773c990-30c8-4335-924d-ad62c009b313 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:40.169436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 261/520] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:40.170699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17d2ec-e495-4cc8-aa77-7b89cd41d03a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.179349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.198474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac01c-5600-4907-8add-2859dd44afa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.199088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac01c-5600-4907-8add-2859dd44afa7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.199362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac01c-5600-4907-8add-2859dd44afa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.199633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac01c-5600-4907-8add-2859dd44afa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.260646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.268522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cac01c-5600-4907-8add-2859dd44afa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.269999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cac01c-5600-4907-8add-2859dd44afa7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.273643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17d2ec-e495-4cc8-aa77-7b89cd41d03a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:40.306675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.307305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.307578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.307844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.314526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.315952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:40.328163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:40.329167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:40.329984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 262/521] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:40.330429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.368416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 260/522] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:40.369257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac01c-5600-4907-8add-2859dd44afa7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.396200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.435652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e453b-9de3-42f0-95a9-3a2675754a3f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:40.463499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.469925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc24fe5-d73e-46cb-9265-e692e4a5540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.470484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac01c-5600-4907-8add-2859dd44afa7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:40.470581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc24fe5-d73e-46cb-9265-e692e4a5540d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.470781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc24fe5-d73e-46cb-9265-e692e4a5540d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.471044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc24fe5-d73e-46cb-9265-e692e4a5540d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.503838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b8976-91a2-4b76-b610-72d201cafac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.504433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b8976-91a2-4b76-b610-72d201cafac6 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.504703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b8976-91a2-4b76-b610-72d201cafac6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.504969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920b8976-91a2-4b76-b610-72d201cafac6 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.505877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b8976-91a2-4b76-b610-72d201cafac6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.507298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920b8976-91a2-4b76-b610-72d201cafac6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.527289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 261/523] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:35:40 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:35:40.528073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b8976-91a2-4b76-b610-72d201cafac6 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.538298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc24fe5-d73e-46cb-9265-e692e4a5540d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.539775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc24fe5-d73e-46cb-9265-e692e4a5540d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.612862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.644336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920b8976-91a2-4b76-b610-72d201cafac6 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:40.655684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 263/524] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:40.657475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc24fe5-d73e-46cb-9265-e692e4a5540d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.684042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.684666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.684943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.685249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.698118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.698848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:40.716410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:40.717337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:40.718212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:40.718212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:40.718212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:40.718212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 262/525] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:40.718426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.721267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.778801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc24fe5-d73e-46cb-9265-e692e4a5540d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:40.814462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.815322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.815597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.815868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.816561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:40.816870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.818340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.831668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7efc7d-2fc9-4482-b919-669b86f3e5ad None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:40.842196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 264/526] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:40.842196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.867626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.869880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.870768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.871130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.871403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.883389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:40.885572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:40.903852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:40.904764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:40.905709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:40.905709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:40.905709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:40.907381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 263/527] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:40.908163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:40.951138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdb246-21b9-49f4-9aeb-94e62f085388 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:40.970479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:40.986736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:40.987367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:40.987643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:40.987908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:40.998604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.000040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:41.014545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:41.015593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:41.016828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:41.016828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:41.016828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:41.020192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 265/528] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:41.021026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.023274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3909cf4-62dc-44f4-8764-7cee7c7adbc3 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.061769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.062738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.063197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.063644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.064578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:41.065077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.066712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.080020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 264/529] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:41.080842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.106129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:41.129196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10d30c-b32c-4bf5-901c-2a65021eec94 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.163394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.163989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.164262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.164533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.165811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:41.166111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.167530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.173739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 266/530] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/role_assignments?user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:41.174532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.208253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bd84-4f7a-44f4-9284-1f2155958a9b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.226238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:41.251187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.251833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.252115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.252537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.253224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:41.253522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.255148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.273528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 265/531] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:41.274571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.307570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d9124-c676-4435-bfff-23a6fdddb625 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.348728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.349711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.349711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.349924 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.350540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:41.350844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.352227 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.370854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 267/532] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:41.370854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.373896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:41.406855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99a3e3-8eee-4a55-a275-f6a4572ee96d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.455844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.456840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.457544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.457727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.467438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.468806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:41.482185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:41.483045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:41.483873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:41.483873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:41.483873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:41.483873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:41.483873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:41.483873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:41.483873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:41.483873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:41.487927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 266/533] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:41.488893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.492355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2ade2-5058-4e50-8ccb-c19f6333a6c2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.522974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:41.535808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.537406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.537997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.538472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.547634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.549631 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:41.564149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:41.565004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:41.565872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:41.565872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:41.565872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:41.565872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:41.565872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:41.565872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:41.565872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:41.565872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:41.566255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 268/534] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:41.567157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.593373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fe6ed-4b59-481a-a4a5-b9ba5702fbc9 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.630140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.631025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.631300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.631571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.638577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.640029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.646350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:41.653022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:41.653872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:41.654765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:41.654765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:41.654765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:41.654765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:41.654765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:41.654765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:41.654765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:41.654765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:41.658905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 267/535] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:41.659741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.698456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834ac6-3412-4120-8fcd-b667a12f14d1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.740356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.741342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.741831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.742324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.743812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:41.744323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.745989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.760955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:41.763608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 269/536] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:41.764927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.768656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe56b1-c2bb-4abb-8c93-b87527f37e9e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.807685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.808314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.808592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.808912 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.809685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:41.809992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.811409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.830672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 268/537] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:41.831528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.882900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8274403-323c-49e0-a670-570a7a017b1f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.911448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:41.925509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.926454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.926931 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.927560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.928920 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:41.929467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.930957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.940326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2453e0-a3db-4946-8c47-86a8b76aa03e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:41.945054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 270/538] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:41.946791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:41.977572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:41.978159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:41.978436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:41.978704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:41.979440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=df27a2ce80b94237bd2b9ecfe35109cb) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:41.979734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:41.981272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:41.987107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 269/539] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/role_assignments?user.id=df27a2ce80b94237bd2b9ecfe35109cb&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:41.987762 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:42.008207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:42.112381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577c6c6-56e0-4532-9b0f-0a70fdd4c2aa None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:42.157269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738bf36-4ad8-4e5c-8ce0-7641bd6cb407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:42.158189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738bf36-4ad8-4e5c-8ce0-7641bd6cb407 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:42.158471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738bf36-4ad8-4e5c-8ce0-7641bd6cb407 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:42.158744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738bf36-4ad8-4e5c-8ce0-7641bd6cb407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:42.159496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738bf36-4ad8-4e5c-8ce0-7641bd6cb407 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=029447e782484e8f8a704b683ede1819) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:42.159805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738bf36-4ad8-4e5c-8ce0-7641bd6cb407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:42.160087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8ae05-b827-46c0-ae4f-72c14e0d6b86 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:42.161304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738bf36-4ad8-4e5c-8ce0-7641bd6cb407 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:42.169900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 271/540] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/role_assignments?user.id=029447e782484e8f8a704b683ede1819&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:42.218691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d78f4e-832d-4f77-9876-8731512f7667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:42.219304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d78f4e-832d-4f77-9876-8731512f7667 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:42.219576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d78f4e-832d-4f77-9876-8731512f7667 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:42.219851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d78f4e-832d-4f77-9876-8731512f7667 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:42.220585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d78f4e-832d-4f77-9876-8731512f7667 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=341277e7e3144e5ea93fa657916821e1) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:42.220889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d78f4e-832d-4f77-9876-8731512f7667 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:42.222427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d78f4e-832d-4f77-9876-8731512f7667 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:42.230371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 270/541] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:41 2022] GET /identity/v3/role_assignments?user.id=341277e7e3144e5ea93fa657916821e1&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:42.901882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e1996-80a5-4a1a-a8d4-e13145456ee9 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:42.902315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e1996-80a5-4a1a-a8d4-e13145456ee9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:42.902411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e1996-80a5-4a1a-a8d4-e13145456ee9 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:42.904028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 272/542] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:42.909344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514103-525e-449e-809d-4f0a10f1d870 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:42.909742 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514103-525e-449e-809d-4f0a10f1d870 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:42.909904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514103-525e-449e-809d-4f0a10f1d870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:42.932570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1e514103-525e-449e-809d-4f0a10f1d870 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:42.933804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1e514103-525e-449e-809d-4f0a10f1d870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:43.022870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 271/543] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:43.408488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e1996-80a5-4a1a-a8d4-e13145456ee9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:43.458956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:43.519094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e1996-80a5-4a1a-a8d4-e13145456ee9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:43.556427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eebd39-0060-4fa2-9fbd-b1bfdc5fa71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:43.557088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eebd39-0060-4fa2-9fbd-b1bfdc5fa71c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:43.557364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eebd39-0060-4fa2-9fbd-b1bfdc5fa71c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:43.557638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eebd39-0060-4fa2-9fbd-b1bfdc5fa71c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:43.568110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eebd39-0060-4fa2-9fbd-b1bfdc5fa71c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-61eebd39-0060-4fa2-9fbd-b1bfdc5fa71c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:43.574021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:43.574867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:43.576047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 273/544] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:43.581197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e514103-525e-449e-809d-4f0a10f1d870 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:43.595509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:43.681880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e514103-525e-449e-809d-4f0a10f1d870 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:43.727302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6419b-a26f-431b-aaa4-bb00cf471908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:43.727989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6419b-a26f-431b-aaa4-bb00cf471908 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:43.728268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6419b-a26f-431b-aaa4-bb00cf471908 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:43.728539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6419b-a26f-431b-aaa4-bb00cf471908 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:43.729330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6419b-a26f-431b-aaa4-bb00cf471908 None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:43.729637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6419b-a26f-431b-aaa4-bb00cf471908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:43.730998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6419b-a26f-431b-aaa4-bb00cf471908 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:43.744739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 272/545] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:43.751740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eebd39-0060-4fa2-9fbd-b1bfdc5fa71c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:43.849245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4b201-1aba-4365-9d96-580a86dfe284 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:43.849565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4b201-1aba-4365-9d96-580a86dfe284 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:43.849798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed4b201-1aba-4365-9d96-580a86dfe284 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:43.851302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 273/546] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:43.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516de88-010b-4a3d-887d-c25baf0244ae None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:43.856728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516de88-010b-4a3d-887d-c25baf0244ae None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:43.856997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516de88-010b-4a3d-887d-c25baf0244ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:43.885900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f516de88-010b-4a3d-887d-c25baf0244ae None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:43.887117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f516de88-010b-4a3d-887d-c25baf0244ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:43.894859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eebd39-0060-4fa2-9fbd-b1bfdc5fa71c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:43.909010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:43.945651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:43.946730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:43.947006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:43.947274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:43.971466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:43.972927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:43.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:43.994722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:43.995548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:43.995548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:43.995548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:43.996571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 274/547] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:44.006899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:44.027478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 274/548] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:44.119152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6c62a-6dc8-49d6-a6ec-6fd65c7015e9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:44.161646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:44.162680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:44.163159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:44.163624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:44.164551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:44.166022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:44.167884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:44.181345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 275/549] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:44.187198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516de88-010b-4a3d-887d-c25baf0244ae None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:44.191229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:44.311308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516de88-010b-4a3d-887d-c25baf0244ae None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:44.359564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:44.360609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:44.361111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:44.361631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] PATH_INFO: `/v3/users/swift` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:44.373170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:44.374895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:44.389842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:44.391279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:35:44.392350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:44.400036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:44.401825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 275/550] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:35:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:44.407991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:44.432912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:44.478093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:44.519073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e63db-9969-43f5-9d14-674c28d7f47b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:44.550157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7187a-b12b-49f1-bc8a-2daee982166f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:44.566951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62af28-2c67-48ef-81df-b3b30e1b64b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:44.568138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62af28-2c67-48ef-81df-b3b30e1b64b8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:44.569041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62af28-2c67-48ef-81df-b3b30e1b64b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:44.569559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62af28-2c67-48ef-81df-b3b30e1b64b8 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:44.570581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62af28-2c67-48ef-81df-b3b30e1b64b8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:44.571198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62af28-2c67-48ef-81df-b3b30e1b64b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:44.574173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62af28-2c67-48ef-81df-b3b30e1b64b8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:44.586757 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a0c22-b2a2-4f2f-9c7a-4a73ca724ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:44.587467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0c22-b2a2-4f2f-9c7a-4a73ca724ef0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:44.588578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0c22-b2a2-4f2f-9c7a-4a73ca724ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:44.588578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a0c22-b2a2-4f2f-9c7a-4a73ca724ef0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:44.604521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 276/551] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 01:35:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:44.605395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a0c22-b2a2-4f2f-9c7a-4a73ca724ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:44.606425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9a8c9-aad8-49c2-a2f7-8697496c34c6 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:44.607016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9a8c9-aad8-49c2-a2f7-8697496c34c6 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:44.607471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9a8c9-aad8-49c2-a2f7-8697496c34c6 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:44.609180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 277/552] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:44.610127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9a8c9-aad8-49c2-a2f7-8697496c34c6 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-144a0c22-b2a2-4f2f-9c7a-4a73ca724ef0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:44.610859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:44.611925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:44.612766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:44.617837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 276/553] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:44.618940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98471e-7f08-4eae-8165-eca0a9e12fcf None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:44.619335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98471e-7f08-4eae-8165-eca0a9e12fcf None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:44.619676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98471e-7f08-4eae-8165-eca0a9e12fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:44.639208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ae98471e-7f08-4eae-8165-eca0a9e12fcf None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:44.640528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ae98471e-7f08-4eae-8165-eca0a9e12fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:44.710812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 277/554] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:44.711948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98471e-7f08-4eae-8165-eca0a9e12fcf None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:44.722571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:44.766877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9a8c9-aad8-49c2-a2f7-8697496c34c6 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:44.811800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:44.817685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98471e-7f08-4eae-8165-eca0a9e12fcf None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:44.854354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bfbcd-4658-4b50-8e4a-1cba97e861ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:44.854565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:44.854972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bfbcd-4658-4b50-8e4a-1cba97e861ca None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:44.855248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bfbcd-4658-4b50-8e4a-1cba97e861ca None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:44.855349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:44.855520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bfbcd-4658-4b50-8e4a-1cba97e861ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:44.855657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:44.856142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:44.856875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:44.857275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:44.858707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:44.872671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 278/555] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:44.873974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:44.955831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:44.994930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddefc78-ff70-4f65-8ba0-8b4019b38b98 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:44.999129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bfbcd-4658-4b50-8e4a-1cba97e861ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.000601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bfbcd-4658-4b50-8e4a-1cba97e861ca None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.030898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc15e5-ec29-41bf-ba46-cadb68827b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.031553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc15e5-ec29-41bf-ba46-cadb68827b1e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.031829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc15e5-ec29-41bf-ba46-cadb68827b1e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.032096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc15e5-ec29-41bf-ba46-cadb68827b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.044336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc15e5-ec29-41bf-ba46-cadb68827b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.045945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc15e5-ec29-41bf-ba46-cadb68827b1e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-7fbc15e5-ec29-41bf-ba46-cadb68827b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:45.062129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:45.063113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:45.063965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:45.063965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:45.063965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:45.067797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 279/556] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:45.069107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07ef40-cd00-4df7-a966-2c547fafbfff None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.069356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07ef40-cd00-4df7-a966-2c547fafbfff None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.069624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07ef40-cd00-4df7-a966-2c547fafbfff None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.070887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 280/557] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:45.072716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07ef40-cd00-4df7-a966-2c547fafbfff None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.113851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 278/558] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:45.115351 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bfbcd-4658-4b50-8e4a-1cba97e861ca None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.142330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:45.186844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07ef40-cd00-4df7-a966-2c547fafbfff None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.208475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:45.216946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bfbcd-4658-4b50-8e4a-1cba97e861ca None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.221784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247959f-bbc4-4348-a6a4-98a07ecefd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.222406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247959f-bbc4-4348-a6a4-98a07ecefd71 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.222677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247959f-bbc4-4348-a6a4-98a07ecefd71 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.222942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247959f-bbc4-4348-a6a4-98a07ecefd71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.234618 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247959f-bbc4-4348-a6a4-98a07ecefd71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-3247959f-bbc4-4348-a6a4-98a07ecefd71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:45.239405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:45.240312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:45.241190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:45.249091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 281/559] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:45.249091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9195d3-74f9-44dc-9823-8e2e6d095822 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.249091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9195d3-74f9-44dc-9823-8e2e6d095822 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.249091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9195d3-74f9-44dc-9823-8e2e6d095822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.250902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.251496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.251766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.252029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.252652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:45.252950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.254318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.264397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-db9195d3-74f9-44dc-9823-8e2e6d095822 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:45.265425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 279/560] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:45.265634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-db9195d3-74f9-44dc-9823-8e2e6d095822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:45.267463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.331277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 282/561] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:45.332509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04197580-de2d-4b1c-a8ce-0f3f752481ae None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.332794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04197580-de2d-4b1c-a8ce-0f3f752481ae None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.333087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04197580-de2d-4b1c-a8ce-0f3f752481ae None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.334335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 283/562] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:45.335018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04197580-de2d-4b1c-a8ce-0f3f752481ae None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.365611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:45.386267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38daf8-0817-4342-9aac-1dd1de814767 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.420494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.421136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.421413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.421682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.432541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.433975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.444301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04197580-de2d-4b1c-a8ce-0f3f752481ae None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:45.448845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:45.449946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:45.450843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:45.450843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:45.450843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:45.450843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 280/563] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:45.450843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.463557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:45.486877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69216a-0030-4802-aa78-7ba63e158ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.487477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69216a-0030-4802-aa78-7ba63e158ff1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.487751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69216a-0030-4802-aa78-7ba63e158ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.488012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69216a-0030-4802-aa78-7ba63e158ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.488652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69216a-0030-4802-aa78-7ba63e158ff1 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:45.488948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69216a-0030-4802-aa78-7ba63e158ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.490325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69216a-0030-4802-aa78-7ba63e158ff1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.498876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 284/564] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:45.500070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db413733-0bb3-44ea-8fc6-26c1654a0ee9 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.500350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db413733-0bb3-44ea-8fc6-26c1654a0ee9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.500617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db413733-0bb3-44ea-8fc6-26c1654a0ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.519234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-db413733-0bb3-44ea-8fc6-26c1654a0ee9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:45.520401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-db413733-0bb3-44ea-8fc6-26c1654a0ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:45.557432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dc7f3-ec3c-44cf-bbdc-91c6a4286d8d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.588978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 285/565] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:45.590084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db413733-0bb3-44ea-8fc6-26c1654a0ee9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.597649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.598690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.598969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.599239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.601369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:45.608920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.610832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:45.622463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:45.623377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 281/566] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:45.624334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.700048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db413733-0bb3-44ea-8fc6-26c1654a0ee9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.703624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:45.728631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058d91e-c023-4517-97bc-db114d0260c7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.737171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.737884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.738258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.738632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.739368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:45.739769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.741269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.757648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 286/567] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:45.757648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.767413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.768151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.768528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.768897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.780050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.782095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:45.799728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:45.800800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:45.801869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:45.801869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:45.801869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:45.807800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 282/568] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:45.807800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.855541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:45.864122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979992d-b5f8-426b-886d-4f5bd89d8867 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.898741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd793-9fe7-4aef-8682-2ee4f40c4cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.899398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd793-9fe7-4aef-8682-2ee4f40c4cc2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.899670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd793-9fe7-4aef-8682-2ee4f40c4cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.899936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fd793-9fe7-4aef-8682-2ee4f40c4cc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.910334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fd793-9fe7-4aef-8682-2ee4f40c4cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:45.913456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162823d-9b11-41ef-ab10-6efc4364907f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-fd9fd793-9fe7-4aef-8682-2ee4f40c4cc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:45.915614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:45.916630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:45.917660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:45.921488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 287/569] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:45.922340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd793-9fe7-4aef-8682-2ee4f40c4cc2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:45.947020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:45.949375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:45.949977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:45.950252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:45.950518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:45.951244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:45.951548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:45.952962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:45.969935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 283/570] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:45.970722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.023178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fd793-9fe7-4aef-8682-2ee4f40c4cc2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.058014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.058822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.059091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.059354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.065624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.067047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.069384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab643f9d-58d1-4fe1-ad37-de5f2029d639 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.072214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:46.078719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:46.079619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:46.080464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 288/571] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:46.080955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.107666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.108195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.108466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.108734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.109390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:46.109684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.111027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.122745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 284/572] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:46.123528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.164074 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.181051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3fe73-f497-47d9-aaaa-d18f96b0a9e6 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.214055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f1338-443d-4d31-baf6-bcfa1a883f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.214707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f1338-443d-4d31-baf6-bcfa1a883f9b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.214988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f1338-443d-4d31-baf6-bcfa1a883f9b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.215355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f1338-443d-4d31-baf6-bcfa1a883f9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.221822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc2c8-de09-47a1-b472-30b9504ee832 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.225307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f1338-443d-4d31-baf6-bcfa1a883f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-1b8f1338-443d-4d31-baf6-bcfa1a883f9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:46.231253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:46.232328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 289/573] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:46.233564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f1338-443d-4d31-baf6-bcfa1a883f9b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.260358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.260955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.261259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.261528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.262173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:46.262471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.263807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.271416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 285/574] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:46.271416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.298504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.339463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f1338-443d-4d31-baf6-bcfa1a883f9b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.371014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef37805-16bc-47de-9792-8ed21afd4600 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.372925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.373580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.373855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.374120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.385646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.387076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.389560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:46.402349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:46.403357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:46.404225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:46.404225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:46.404225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:46.406235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.406837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.407118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.407386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.407910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 290/575] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:46.408261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:46.408565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.408707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.410031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.422569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 286/576] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:46.423364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.506531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.511567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad38160-1adf-415f-a5a0-b171e5563f0a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.521910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95b8f9-3854-4c37-a649-c1d39684680e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.544057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.544667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.544940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.545239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.552307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.553761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.554978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.556105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.556384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.556655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.557333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:46.557634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.558965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:46.565106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:46.566056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:46.572141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 291/577] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:46.572831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 287/578] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:46.572912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.572990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.624868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.668154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c25b60-6631-4fb0-9ec0-e611b4b69982 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.668404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e67be-b6f5-453a-a5da-bac5461a3f63 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.700592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.701240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.701345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.701518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.701787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.702077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.702416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.702694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.703316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:46.703780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.705470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.712448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.713885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.718053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 288/579] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:46.718824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.729417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:46.729554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:46.730582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:46.731463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:46.731463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:46.731463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:46.731463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 292/580] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:46.731774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.819245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe7796-099c-4da5-bbb2-07d27880a1e8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.829563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5da8-b433-425a-a299-d3aa33f6cc95 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:46.839244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.852914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.853555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.853835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.854134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.854848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=e0d08965bc9c4ca29812d304bfb4ab4f) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:46.855157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.856576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.861851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 289/581] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/role_assignments?user.id=e0d08965bc9c4ca29812d304bfb4ab4f&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:46.862621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.864717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:46.865468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:46.865820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:46.866842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:46.878789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:46.880322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:46.903274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:46.904454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:46.905457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:46.905457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:46.905457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:46.905457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 293/582] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:46.905865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:46.947030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:46.978490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ec8c3-f7ba-4806-bfb1-0c3d03217d47 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.014117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.014733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.015012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.015285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.016026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:47.016330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.017847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.027459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df1bbd-dfe1-4d78-b16a-b4ed9405dcb1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.037502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 290/583] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:47.038382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.066752 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.067488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.067835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.068231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.075141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.076657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:47.090004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:47.091063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:47.092137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:47.092137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:47.092137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:47.092137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:47.092137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:47.092137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:47.092137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:47.092137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:47.092851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 294/584] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:47.097963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.121723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:47.155380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf71b3-cdc2-4e9f-8503-a941a88e7884 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.190819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:47.198741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.199892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.200493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.200964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.202261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:47.202809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.204613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.206168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a823dc3-48f1-4fbd-82fc-80037fbf4973 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.222148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 291/585] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:47.222990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.244004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.244670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.244965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.245267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.245906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:47.246209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.247585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.261911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 295/586] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:47.262555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.327689 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e5e99-3c53-4666-850a-78c6e830bca1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.348452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:47.362684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.363355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.363632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.363905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.370513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91219f7-6686-4866-8f04-c51b3dd92df5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.374966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.376398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:47.390935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:47.392071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:47.393229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:47.393229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:47.393229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:47.396719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 292/587] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:47.397611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.410127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.410781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.411050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.411316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.411990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:47.412291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.413965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.427336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 296/588] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:47.428262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.452310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:47.513584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811c19-72c3-4ac6-b6bc-9bde2794e1c3 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.549975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6effb-7157-4b31-99b1-f3f1c33062a9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.550574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.551210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.551529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.551751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] PATH_INFO: `/v3/users/glance` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.560681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.562209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:47.574070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:47.575001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 293/589] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:47.576003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.591735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.592371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.592643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.592910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.600634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.602082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.605179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:47.615050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:47.615994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 297/590] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:47.622261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.687170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f9109-defc-4505-b6f9-90998d1071ab None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.688611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:47.722672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e897c-1e32-4569-8975-c290e3e87f00 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.725137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.725792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.726072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.726375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.727063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:47.727410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.728987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.746674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 294/591] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:47.746674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.767577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cc8db-460b-48a8-a0d1-4261b5628dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.768299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cc8db-460b-48a8-a0d1-4261b5628dad None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.768580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cc8db-460b-48a8-a0d1-4261b5628dad None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.768850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cc8db-460b-48a8-a0d1-4261b5628dad None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/5a63d93adb4e4044ae79fb5aa7d8f5e3/roles/a48f239bb178461d9d0ab9daaef7e154` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.783643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cc8db-460b-48a8-a0d1-4261b5628dad None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=5a63d93adb4e4044ae79fb5aa7d8f5e3, role_id=a48f239bb178461d9d0ab9daaef7e154)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.785079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6cc8db-460b-48a8-a0d1-4261b5628dad None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.799865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 298/592] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:35:47 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/5a63d93adb4e4044ae79fb5aa7d8f5e3/roles/a48f239bb178461d9d0ab9daaef7e154 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:47.801337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cc8db-460b-48a8-a0d1-4261b5628dad None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.864541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae3fdc-368e-4da4-a690-914ce9dcec24 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.886023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:47.925608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:47.926238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:47.926512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:47.926782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:47.927523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:47.927829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:47.929312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:47.950728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cc8db-460b-48a8-a0d1-4261b5628dad None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:47.950910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 295/593] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:47.951918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:47.976634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:48.004125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.005116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.005859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.006362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.007721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:48.008237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.010468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.057143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 299/594] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:48.058716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:48.077446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627719b-0083-433f-bf9b-6797abf10d8c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.122226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.123105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.123534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.123976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.131583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.133219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:48.145143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:48.146267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:48.147345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:48.147345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:48.147345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:48.147345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:48.147345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:48.147345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:48.147345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:48.147345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:48.149926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 296/595] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:48.155270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:48.180955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2504d2-9faf-4ebf-b5ba-62c72926812d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.197128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:48.233756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.234323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.234604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.234877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.248549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.250145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.250821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:48.270566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:48.271574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:48.272469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:48.272469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:48.272469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:48.272469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 300/596] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:48.273137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:48.275703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7611c-3b27-4be7-8ba5-b6a398acda38 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.311233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.312241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.312723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.313284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.324736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.326533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:48.342362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:48.343300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:48.344261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:48.344261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:48.344261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:48.350946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 297/597] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:48.352052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:48.402997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b4a89-3c77-41ff-8327-2b39dc1eacf8 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.450164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.450775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.451051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.451325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.452134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:48.452444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.454336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.473162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67197e-0d04-4890-852c-6d0b3ba3105e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.474105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 301/598] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:48.474976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:48.497590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:48.514220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:48.519420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.519735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.519941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.520218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.520883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:48.521216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.522750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.536716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 298/599] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:48.537537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:48.609976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a0447-8700-4371-a87d-d950b73d55e0 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.645150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcccccb-1cd5-45c8-90fb-9ac3763cfdf8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.657823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.658985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.659424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.659852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.660914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:48.661405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.663678 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.680412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 302/600] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:48.681632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:48.688074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b82058-b4d2-456b-ae55-cc1fb36e7e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.689032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82058-b4d2-456b-ae55-cc1fb36e7e10 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.689235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82058-b4d2-456b-ae55-cc1fb36e7e10 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.689611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82058-b4d2-456b-ae55-cc1fb36e7e10 None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/df27a2ce80b94237bd2b9ecfe35109cb/roles/a48f239bb178461d9d0ab9daaef7e154` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.708687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b82058-b4d2-456b-ae55-cc1fb36e7e10 None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=df27a2ce80b94237bd2b9ecfe35109cb, role_id=a48f239bb178461d9d0ab9daaef7e154)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.710153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b82058-b4d2-456b-ae55-cc1fb36e7e10 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.728514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 299/601] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:35:48 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/df27a2ce80b94237bd2b9ecfe35109cb/roles/a48f239bb178461d9d0ab9daaef7e154 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:48.733954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b82058-b4d2-456b-ae55-cc1fb36e7e10 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:48.828513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:48.833960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:48.854353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9da67-6984-419d-9058-42c6cf997990 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.927556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:48.928222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:48.928494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:48.928761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:48.944435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:48.948248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:48.965692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b82058-b4d2-456b-ae55-cc1fb36e7e10 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:48.971336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:48.972230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:48.973165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 303/602] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:48.979533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:49.017842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186189f-613b-4af7-8046-fc2573e2ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:49.018496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186189f-613b-4af7-8046-fc2573e2ff7e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:49.018773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186189f-613b-4af7-8046-fc2573e2ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:49.019044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186189f-613b-4af7-8046-fc2573e2ff7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:49.027818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186189f-613b-4af7-8046-fc2573e2ff7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:49.029387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186189f-613b-4af7-8046-fc2573e2ff7e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-3186189f-613b-4af7-8046-fc2573e2ff7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:49.042791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:49.043889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:49.044793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 300/603] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:49.054763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b457f4-dba6-47d1-b5de-847e7f2f5886 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:49.054763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b457f4-dba6-47d1-b5de-847e7f2f5886 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:49.054763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b457f4-dba6-47d1-b5de-847e7f2f5886 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:49.054763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 301/604] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:49.054763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b457f4-dba6-47d1-b5de-847e7f2f5886 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:49.104645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6b3ab-b54a-4df0-b1b3-76a1a67a30a6 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:49.146410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e2c69-ed8b-492b-8e44-4231e4bc83b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:49.147044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e2c69-ed8b-492b-8e44-4231e4bc83b7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:49.147327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e2c69-ed8b-492b-8e44-4231e4bc83b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:49.147603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e2c69-ed8b-492b-8e44-4231e4bc83b7 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:49.148299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e2c69-ed8b-492b-8e44-4231e4bc83b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:49.148606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e2c69-ed8b-492b-8e44-4231e4bc83b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:49.150910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e2c69-ed8b-492b-8e44-4231e4bc83b7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:49.163416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:49.171711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 304/605] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:49.174060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91984110-ed01-4b95-9394-aa972fc2f8c2 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:49.174350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91984110-ed01-4b95-9394-aa972fc2f8c2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:49.174624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91984110-ed01-4b95-9394-aa972fc2f8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:49.190094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b457f4-dba6-47d1-b5de-847e7f2f5886 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:49.192291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:49.201361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-91984110-ed01-4b95-9394-aa972fc2f8c2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:49.204324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-91984110-ed01-4b95-9394-aa972fc2f8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:49.232673 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:49.233622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:49.234120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:49.234588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:49.235433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:49.235881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:49.237696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:49.251341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 302/606] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:49.252839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:49.301195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 305/607] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:49.302331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91984110-ed01-4b95-9394-aa972fc2f8c2 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:49.378060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e56a-27cb-420a-b46f-21e653c05c90 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:49.417343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd172d1e-d05f-4fa6-aae0-c38ffcbd6ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:49.418145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172d1e-d05f-4fa6-aae0-c38ffcbd6ccf None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:49.418427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172d1e-d05f-4fa6-aae0-c38ffcbd6ccf None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:49.418702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172d1e-d05f-4fa6-aae0-c38ffcbd6ccf None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/029447e782484e8f8a704b683ede1819/roles/a48f239bb178461d9d0ab9daaef7e154` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:49.435765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd172d1e-d05f-4fa6-aae0-c38ffcbd6ccf None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=029447e782484e8f8a704b683ede1819, role_id=a48f239bb178461d9d0ab9daaef7e154)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:49.437553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd172d1e-d05f-4fa6-aae0-c38ffcbd6ccf None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:49.462449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 303/608] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:35:49 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/029447e782484e8f8a704b683ede1819/roles/a48f239bb178461d9d0ab9daaef7e154 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:49.494636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91984110-ed01-4b95-9394-aa972fc2f8c2 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:49.529577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:49.537481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f255e3-45f4-4f3d-9e07-ee2cf4f0a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:49.538360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f255e3-45f4-4f3d-9e07-ee2cf4f0a908 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:49.538794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f255e3-45f4-4f3d-9e07-ee2cf4f0a908 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:49.539233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f255e3-45f4-4f3d-9e07-ee2cf4f0a908 None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/341277e7e3144e5ea93fa657916821e1/roles/a48f239bb178461d9d0ab9daaef7e154` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:49.556374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f255e3-45f4-4f3d-9e07-ee2cf4f0a908 None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=341277e7e3144e5ea93fa657916821e1, role_id=a48f239bb178461d9d0ab9daaef7e154)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:49.558282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f255e3-45f4-4f3d-9e07-ee2cf4f0a908 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:49.581626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 306/609] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:35:49 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/341277e7e3144e5ea93fa657916821e1/roles/a48f239bb178461d9d0ab9daaef7e154 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:49.722711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd172d1e-d05f-4fa6-aae0-c38ffcbd6ccf None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:49.740122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:49.870109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd172d1e-d05f-4fa6-aae0-c38ffcbd6ccf None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:49.931450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6c8be-0649-4bd9-a4c8-c2433178921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:49.932098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c8be-0649-4bd9-a4c8-c2433178921f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:49.932379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c8be-0649-4bd9-a4c8-c2433178921f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:49.932647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c8be-0649-4bd9-a4c8-c2433178921f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:49.949525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6c8be-0649-4bd9-a4c8-c2433178921f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-09a6c8be-0649-4bd9-a4c8-c2433178921f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:49.954619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:49.955527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:49.956333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 304/610] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:49.962113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f255e3-45f4-4f3d-9e07-ee2cf4f0a908 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:50.091325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f255e3-45f4-4f3d-9e07-ee2cf4f0a908 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:50.132513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf3007-4e66-4059-b72c-5da737ef2748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:50.133295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3007-4e66-4059-b72c-5da737ef2748 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:50.134004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3007-4e66-4059-b72c-5da737ef2748 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:50.134285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3007-4e66-4059-b72c-5da737ef2748 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:50.134948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf3007-4e66-4059-b72c-5da737ef2748 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:50.135250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf3007-4e66-4059-b72c-5da737ef2748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:50.136615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf3007-4e66-4059-b72c-5da737ef2748 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:50.145346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 307/611] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:50.150769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6c8be-0649-4bd9-a4c8-c2433178921f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:50.209139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a02da5-15e3-4a2c-9d4a-fa24bcfab84e None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:50.209139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a02da5-15e3-4a2c-9d4a-fa24bcfab84e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:50.209139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a02da5-15e3-4a2c-9d4a-fa24bcfab84e None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:50.210293 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 308/612] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:50.215159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f838e9a-f661-4dee-9f6e-6222bc885339 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:50.215539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f838e9a-f661-4dee-9f6e-6222bc885339 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:50.215908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f838e9a-f661-4dee-9f6e-6222bc885339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:50.247651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-8f838e9a-f661-4dee-9f6e-6222bc885339 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:50.248830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8f838e9a-f661-4dee-9f6e-6222bc885339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:50.271247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:50.308653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:50.335912 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6c8be-0649-4bd9-a4c8-c2433178921f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:50.349099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 309/613] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:35:50.391287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:50.392306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:50.392796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:50.393311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:50.412811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:50.414598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:50.437216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:50.438208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:50.439089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:50.439089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:50.439089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:50.439089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 305/614] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:50.445514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f838e9a-f661-4dee-9f6e-6222bc885339 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:50.593879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f838e9a-f661-4dee-9f6e-6222bc885339 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:50.638467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:50.638986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:50.639258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:50.639525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:50.640162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:50.640459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:50.642090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:50.660020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 310/615] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:50.665344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:50.817004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d56a9-d098-4eb8-bd9b-d7f139bcffbc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:50.820561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:50.848391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:50.870809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:50.872145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:50.874557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:50.874839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:50.875107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:50.891085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:50.894874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:50.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:50.916920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:35:50.917901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:50.921134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 306/616] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:35:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:50.927289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:51.010532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678bfe96-c13e-4b48-b990-0a1cbd203731 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:51.078107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8086f-ca3f-4081-abe5-0bbcbc7bf587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:51.078738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8086f-ca3f-4081-abe5-0bbcbc7bf587 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.079057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8086f-ca3f-4081-abe5-0bbcbc7bf587 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.079327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8086f-ca3f-4081-abe5-0bbcbc7bf587 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.103159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8086f-ca3f-4081-abe5-0bbcbc7bf587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-99f8086f-ca3f-4081-abe5-0bbcbc7bf587 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:51.109122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:51.110022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:51.111387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 311/617] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:51.117249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8086f-ca3f-4081-abe5-0bbcbc7bf587 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:51.139111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432add1-8f34-474c-a050-645f4818efc4 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:51.201888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:51.202514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.202797 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.203071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.203810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:51.204119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:51.206152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:51.255332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 307/618] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 01:35:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:51.260837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:51.261956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:51.270533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8086f-ca3f-4081-abe5-0bbcbc7bf587 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:51.311408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:51.312024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.312307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.312583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.313283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:51.313596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:51.315654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:51.324664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 312/619] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:51.331053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:51.380250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:51.441679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb3b99-8e82-4406-8ad4-18f87c52d485 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:51.487732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:51.488291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.488572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.488844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.503489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:51.504945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:51.520811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:51.521823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:51.522689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:51.522689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:51.522689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:51.522689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 313/620] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:51.530167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:51.553538 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:51.571500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b4de9-0e1e-45ea-983e-a0619bbe20cc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:51.615818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04def82c-da81-46b6-a729-3409dda97397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:51.617104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04def82c-da81-46b6-a729-3409dda97397 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.618574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04def82c-da81-46b6-a729-3409dda97397 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.619062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04def82c-da81-46b6-a729-3409dda97397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.632074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04def82c-da81-46b6-a729-3409dda97397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:51.633833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04def82c-da81-46b6-a729-3409dda97397 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-04def82c-da81-46b6-a729-3409dda97397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:51.648467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:51.649716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:51.650604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:51.650604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:51.650604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:51.650604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 308/621] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:51.651182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc2cf8-d00c-4e7c-adfa-fdf358d608db None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.651467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc2cf8-d00c-4e7c-adfa-fdf358d608db None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.651721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc2cf8-d00c-4e7c-adfa-fdf358d608db None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.661536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 309/622] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:51.661536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc2cf8-d00c-4e7c-adfa-fdf358d608db None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:51.681575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a1cc6-e3a1-4d5c-bfe5-c1378b6e19c8 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:51.719147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86a5e2-feae-4cf0-8568-8303d3464c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:51.720136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86a5e2-feae-4cf0-8568-8303d3464c0c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.720727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86a5e2-feae-4cf0-8568-8303d3464c0c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.721354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86a5e2-feae-4cf0-8568-8303d3464c0c None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.722222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86a5e2-feae-4cf0-8568-8303d3464c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:51.722721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86a5e2-feae-4cf0-8568-8303d3464c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:51.724558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86a5e2-feae-4cf0-8568-8303d3464c0c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:51.738900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 314/623] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:51.740539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39e09-1df4-48b9-8594-bf4199f6f82b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.741032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39e09-1df4-48b9-8594-bf4199f6f82b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.741551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39e09-1df4-48b9-8594-bf4199f6f82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.761837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-84d39e09-1df4-48b9-8594-bf4199f6f82b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:51.762808 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:51.764679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-84d39e09-1df4-48b9-8594-bf4199f6f82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:51.771530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc2cf8-d00c-4e7c-adfa-fdf358d608db None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:51.807292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:51.807913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:51.808197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:51.808459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:51.809432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:51.809432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:51.810796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:51.827812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 310/624] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:51.828697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:51.831682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:51.845144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 315/625] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:51.846310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d39e09-1df4-48b9-8594-bf4199f6f82b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.028787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02044953-8dac-41b7-a197-947617a3fe58 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.070407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.071098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.071223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d39e09-1df4-48b9-8594-bf4199f6f82b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.071425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.071643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.080741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:52.082233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:52.095544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:52.096527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:52.097503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 311/626] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:35:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:52.102994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.120120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:52.136660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb24e1-afd0-4713-9c88-ad1645f86150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.137673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb24e1-afd0-4713-9c88-ad1645f86150 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.138253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb24e1-afd0-4713-9c88-ad1645f86150 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.138845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb24e1-afd0-4713-9c88-ad1645f86150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.146405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb24e1-afd0-4713-9c88-ad1645f86150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:52.148213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb24e1-afd0-4713-9c88-ad1645f86150 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-9cfb24e1-afd0-4713-9c88-ad1645f86150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:52.161498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:52.162448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:52.163321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:52.163321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:52.163321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:52.163321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:52.163321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:52.163321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:52.163321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:52.163321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:52.166436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 316/627] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:52.168073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cb745-5ba6-45c7-b643-861d95cbe645 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.168647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cb745-5ba6-45c7-b643-861d95cbe645 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.169141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cb745-5ba6-45c7-b643-861d95cbe645 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.171007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 317/628] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:52.172386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e834b4-204a-4bc4-83e5-a1d348af86bb None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.172881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e834b4-204a-4bc4-83e5-a1d348af86bb None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.173378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e834b4-204a-4bc4-83e5-a1d348af86bb None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.175236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 318/629] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:52.176677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf16c0-2d1e-4d89-8bfa-d1c91c91efe2 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.177231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf16c0-2d1e-4d89-8bfa-d1c91c91efe2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.177728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf16c0-2d1e-4d89-8bfa-d1c91c91efe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.195741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6cdf16c0-2d1e-4d89-8bfa-d1c91c91efe2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:52.197570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6cdf16c0-2d1e-4d89-8bfa-d1c91c91efe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:52.211352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f44cf9-e122-4cd2-b03a-77a3ad1f7716 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.245194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bafd2e-ef72-4f60-9d04-246794a1d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.245791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bafd2e-ef72-4f60-9d04-246794a1d63a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.246065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bafd2e-ef72-4f60-9d04-246794a1d63a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.246334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bafd2e-ef72-4f60-9d04-246794a1d63a None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.247060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bafd2e-ef72-4f60-9d04-246794a1d63a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:52.247314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:52.247437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bafd2e-ef72-4f60-9d04-246794a1d63a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:52.248804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bafd2e-ef72-4f60-9d04-246794a1d63a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:52.266679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 319/630] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:52.267537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf16c0-2d1e-4d89-8bfa-d1c91c91efe2 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.267681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 312/631] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:35:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:52.269008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d107e-7f24-4cbb-b2b5-aa205cbe737e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.269471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d107e-7f24-4cbb-b2b5-aa205cbe737e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.269875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d107e-7f24-4cbb-b2b5-aa205cbe737e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.288886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-797d107e-7f24-4cbb-b2b5-aa205cbe737e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:52.290095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-797d107e-7f24-4cbb-b2b5-aa205cbe737e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:52.363628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 313/632] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:52.364594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d107e-7f24-4cbb-b2b5-aa205cbe737e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.367100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:52.383964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf16c0-2d1e-4d89-8bfa-d1c91c91efe2 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.426633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.427560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.428008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.428424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.429275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:52.429822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:52.431571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:52.447038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 320/633] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:52.448459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.501776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d107e-7f24-4cbb-b2b5-aa205cbe737e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.504700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:52.548113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19796b6f-96af-4bad-a47c-085e5e546cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.548664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796b6f-96af-4bad-a47c-085e5e546cfd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.548942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796b6f-96af-4bad-a47c-085e5e546cfd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.549258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19796b6f-96af-4bad-a47c-085e5e546cfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.560395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19796b6f-96af-4bad-a47c-085e5e546cfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-19796b6f-96af-4bad-a47c-085e5e546cfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:52.565246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:52.566169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:52.566992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 314/634] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:52.567686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19796b6f-96af-4bad-a47c-085e5e546cfd None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.573292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32593b-ec7a-4591-a099-1415507a72e4 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.610606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa3923-f598-4f11-b794-ab3ac63f171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.611208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa3923-f598-4f11-b794-ab3ac63f171d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.611507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa3923-f598-4f11-b794-ab3ac63f171d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.611775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa3923-f598-4f11-b794-ab3ac63f171d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.629553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:52.668953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19796b6f-96af-4bad-a47c-085e5e546cfd None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.682016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa3923-f598-4f11-b794-ab3ac63f171d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:52.683514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa3923-f598-4f11-b794-ab3ac63f171d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:52.702404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99981581-3619-48d1-b2ca-75136ee41895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.703050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99981581-3619-48d1-b2ca-75136ee41895 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.703326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99981581-3619-48d1-b2ca-75136ee41895 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.703597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99981581-3619-48d1-b2ca-75136ee41895 None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/e0d08965bc9c4ca29812d304bfb4ab4f/roles/a48f239bb178461d9d0ab9daaef7e154` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.717543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99981581-3619-48d1-b2ca-75136ee41895 None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=e0d08965bc9c4ca29812d304bfb4ab4f, role_id=a48f239bb178461d9d0ab9daaef7e154)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:52.718969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99981581-3619-48d1-b2ca-75136ee41895 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:52.728090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:52.731978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 315/635] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:35:52 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/e0d08965bc9c4ca29812d304bfb4ab4f/roles/a48f239bb178461d9d0ab9daaef7e154 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:35:52.732768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99981581-3619-48d1-b2ca-75136ee41895 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.791610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 321/636] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:52.792944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa3923-f598-4f11-b794-ab3ac63f171d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.856392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99981581-3619-48d1-b2ca-75136ee41895 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.871758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:52.904983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f52b0-3632-45f4-8a95-7a664ce7dc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.905814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f52b0-3632-45f4-8a95-7a664ce7dc30 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.906093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f52b0-3632-45f4-8a95-7a664ce7dc30 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.906377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f52b0-3632-45f4-8a95-7a664ce7dc30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.916273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa3923-f598-4f11-b794-ab3ac63f171d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:52.921680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f52b0-3632-45f4-8a95-7a664ce7dc30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-181f52b0-3632-45f4-8a95-7a664ce7dc30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:52.926519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:52.927399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:52.928228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 316/637] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:52.928899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f52b0-3632-45f4-8a95-7a664ce7dc30 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.960495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:52.961751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:52.962529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:52.963024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:52.964067 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:52.964558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:52.965977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:52.974487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 322/638] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:52.975563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:52.988784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:53.044122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f52b0-3632-45f4-8a95-7a664ce7dc30 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.078065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774e1d0-24ee-4eba-b826-b9bc1e9e609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.078692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e1d0-24ee-4eba-b826-b9bc1e9e609e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.078966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e1d0-24ee-4eba-b826-b9bc1e9e609e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.079233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e1d0-24ee-4eba-b826-b9bc1e9e609e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.084950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edfbae-4eb0-4358-824c-5f528d997d0f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.089153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774e1d0-24ee-4eba-b826-b9bc1e9e609e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-e774e1d0-24ee-4eba-b826-b9bc1e9e609e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:53.094110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:53.095255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 317/639] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:53.096440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774e1d0-24ee-4eba-b826-b9bc1e9e609e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.125169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.126133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.126560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:53.126995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.128041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.141112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.142749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:53.158865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:53.160009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:53.161132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:53.161132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:53.161132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:53.163125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 323/640] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:53.164414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.199132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774e1d0-24ee-4eba-b826-b9bc1e9e609e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.235756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.236363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.236697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.236973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.237696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:53.237995 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.239187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:53.239328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.256467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 318/641] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:53.256467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.306164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541658b-1453-454b-b3ff-a245e0a53f2b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.345220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.345839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.346112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.346375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.349400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:53.357699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fa8c3-1786-4037-994e-72ca4a37c80b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.357826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.359191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:53.374194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:53.375246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:53.376135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:53.376135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:53.376135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:53.376135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 324/642] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:53.376135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.397013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.397645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.397918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.398184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.398831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:53.399668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.401016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.409019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 319/643] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:53.409850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.495131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b0ec7-0b8c-49a7-8859-39cecd2a5bf5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.508743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:53.509313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538188f-27fd-42d6-88c5-aa9db44329ce None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.533227 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.533906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.534342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.534676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.535489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:53.535855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.537451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.542940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.543666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.544040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.544409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.553957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 325/644] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:53.553957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.558755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.560258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:53.575161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:53.576256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:53.577533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:53.577533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:53.577533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:53.577533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 320/645] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:53.577533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.598125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:53.674376 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152905f2-c9a6-4d16-82e5-e9de6380d610 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.697393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3e765-c314-465e-a72b-72db9bcd437d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.710507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.711117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.711390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.712292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.712916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:53.713288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.714595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.730344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 326/646] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:53.730344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.736363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.737426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.737715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.737986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.749906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.751116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.760409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:53.766550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:53.767453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:53.768389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:53.768389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:53.768389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:53.772221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 321/647] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:53.773144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.832999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:53.835057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f221ad-83ee-4ebf-9bf0-68173ce8c699 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.870649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.871271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.871549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.871855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.877173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628f7d0-f0da-438a-a5a4-ab0a6ffd5809 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:53.878447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.880177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:53.895263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:53.896320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 327/648] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:53.897877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:53.924177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:53.924928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:53.925247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:53.925593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:53.926304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:53.926681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:53.928063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:53.940276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 322/649] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:53.941201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.004585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:54.016496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b1759-893a-4e8a-9fff-21ab12084612 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.048543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6e8dc-69aa-4757-ad6e-0bd70fdf8f37 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.053909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.054540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.054815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.055082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.062800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.064250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.070546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:54.075418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:54.076324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:54.077639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 328/650] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:54.078228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.091603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.092210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.092486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.092754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.093424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:54.093722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.095080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.107668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 323/651] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:54.108537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.185817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81807e2-8dbc-4576-9e98-228d1f674503 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.211938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09202d08-4658-41aa-8e7f-c1a0f168220e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.219542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.220140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.220411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.221026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.221733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:54.222335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.223755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.242357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 329/652] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:54.242357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.251528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.252275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.252552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.252825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.254573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:54.263520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.265199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:54.278773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:54.279692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 324/653] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:54.280641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.325669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:54.351189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da17fd-57a4-43bc-835a-b22f4a99cdef None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.386899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad5002-e12c-4b4b-a441-4472750ca104 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.387402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.388079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.388426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.388706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.389476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:54.389786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.391198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.408800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 330/654] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:54.414087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.424078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.425178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.425554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.426203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.433982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.435456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:54.447448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:54.448547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 01:35:54.449850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:54.453172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 325/655] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:54.454006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.490447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:54.526476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac865e-c3c6-4e66-b1a7-71505b186a02 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.553267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:54.556948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926aefca-c247-4b32-af2b-c809095987f6 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.562530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.563241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.563614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.563978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.564788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:54.565242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.566741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.573238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 331/656] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/role_assignments?user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:54.574182 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.607057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.607986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.608355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.608725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.609629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:54.610095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.611824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.633111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 326/657] 10.222.0.15 () {56 vars in 1137 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:54.634846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.716661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9ddca1-1ae8-4e92-a847-22ff2b913bd9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.754116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2863d-5807-4eec-b87d-a9908dbff148 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.767471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ca275-17a8-4780-8e20-b48976c88514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.768265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca275-17a8-4780-8e20-b48976c88514 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.768641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca275-17a8-4780-8e20-b48976c88514 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.769003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca275-17a8-4780-8e20-b48976c88514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.773769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:54.806977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:54.807883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:54.808259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:54.808629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:54.809817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:54.810257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.812070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.843567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 327/658] 10.222.0.15 () {56 vars in 1139 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:54.843567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:54.862257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ca275-17a8-4780-8e20-b48976c88514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:54.862617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:54.863928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ca275-17a8-4780-8e20-b48976c88514 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:54.968042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fac7452-6d98-4f7e-9b75-55c4a45a2183 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:54.981159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 332/659] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:54.981778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ca275-17a8-4780-8e20-b48976c88514 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.009523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068838d4-9064-4979-a07b-996effc8c7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:55.010379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068838d4-9064-4979-a07b-996effc8c7e8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.010663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068838d4-9064-4979-a07b-996effc8c7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.010931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068838d4-9064-4979-a07b-996effc8c7e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.018225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:55.083132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068838d4-9064-4979-a07b-996effc8c7e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:55.084801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068838d4-9064-4979-a07b-996effc8c7e8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:55.090055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ca275-17a8-4780-8e20-b48976c88514 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:55.104265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:55.126212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef414bb1-07ce-4ea5-a49a-c12af89143a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:55.126812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef414bb1-07ce-4ea5-a49a-c12af89143a8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.127083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef414bb1-07ce-4ea5-a49a-c12af89143a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.127410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef414bb1-07ce-4ea5-a49a-c12af89143a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.197941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 328/660] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:55.199542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16eeb7-0aea-47e5-97f6-2e15c2eb125d None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.200026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16eeb7-0aea-47e5-97f6-2e15c2eb125d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.200496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16eeb7-0aea-47e5-97f6-2e15c2eb125d None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.202241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 329/661] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:55.203171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16eeb7-0aea-47e5-97f6-2e15c2eb125d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.209539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef414bb1-07ce-4ea5-a49a-c12af89143a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:55.211009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef414bb1-07ce-4ea5-a49a-c12af89143a8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:55.252047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:55.325435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 333/662] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:55.325435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef414bb1-07ce-4ea5-a49a-c12af89143a8 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.331191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16eeb7-0aea-47e5-97f6-2e15c2eb125d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:55.349163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:55.368721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3a2e4-4980-44c0-aee1-4376c2f66264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:55.369462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a2e4-4980-44c0-aee1-4376c2f66264 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.369800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a2e4-4980-44c0-aee1-4376c2f66264 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.370142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a2e4-4980-44c0-aee1-4376c2f66264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.381274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3a2e4-4980-44c0-aee1-4376c2f66264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:55.382839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3a2e4-4980-44c0-aee1-4376c2f66264 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-05b3a2e4-4980-44c0-aee1-4376c2f66264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:55.398382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:55.399575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:55.400712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:55.400712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:55.400712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:55.406137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 330/663] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:55.406137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c3b4d-3920-4797-8e96-4e0673db91b8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.406137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c3b4d-3920-4797-8e96-4e0673db91b8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.406137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c3b4d-3920-4797-8e96-4e0673db91b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.422919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0d2c3b4d-3920-4797-8e96-4e0673db91b8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:55.424191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0d2c3b4d-3920-4797-8e96-4e0673db91b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:55.430814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef414bb1-07ce-4ea5-a49a-c12af89143a8 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:55.466037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:55.466668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.466947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.467206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.469051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:55.478754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:55.480174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:55.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:55.496617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:55.497843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:55.497843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:55.497843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:55.503644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 334/664] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:55.503644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.505329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 331/665] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:55.506338 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c3b4d-3920-4797-8e96-4e0673db91b8 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.628204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:55.634748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c3b4d-3920-4797-8e96-4e0673db91b8 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:55.636430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cfad1a-f132-4e8d-8d23-8eaa985a81c3 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:55.677456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:55.678522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.679014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.679515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.680033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:55.681283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.681653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.682030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.682765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:55.683165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:55.684629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:55.694146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:55.696572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:55.702024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 332/666] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:55.702970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:55.714246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:55.715359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:55.716401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:55.716401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:55.716401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:55.718470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 335/667] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:55.719470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.771998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:55.812886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922897e1-bf22-4db6-894b-cbfbbfa1d643 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:55.835526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ab4d1-27f9-467d-a459-7c486e4e2c9b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:55.850978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:55.851577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.851849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.852145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.852775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:55.853092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:55.854442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:55.866306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 333/668] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:55.867107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.874039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:55.874666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:55.874940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:55.875204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:55.882147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:55.884173 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:55.889608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:55.895790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:55.896677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 336/669] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:55.898191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:55.989678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef1185-bee3-4668-b423-ef3dc77c011d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.011927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:56.016103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08e450-1bbc-4433-ad18-6bc63212fe53 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.025486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.026042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.026320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.026595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.027236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:56.027540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.028905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.041433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 334/670] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:56.042072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.058200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f9b5d-3bfb-434b-bce3-39dce909fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.059091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f9b5d-3bfb-434b-bce3-39dce909fb03 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.059512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f9b5d-3bfb-434b-bce3-39dce909fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.059951 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f9b5d-3bfb-434b-bce3-39dce909fb03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.073340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f9b5d-3bfb-434b-bce3-39dce909fb03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-464f9b5d-3bfb-434b-bce3-39dce909fb03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:56.080021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:56.081088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:35:56.081952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:56.083171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 337/671] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:35:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:56.083819 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f9b5d-3bfb-434b-bce3-39dce909fb03 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.134677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:56.160452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74d234-c127-4fd8-8dcc-4efcb64263d3 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.187538 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f9b5d-3bfb-434b-bce3-39dce909fb03 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.196444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.197167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.197511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.197849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.204528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.206053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:56.217758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:56.218778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:56.219830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:56.219830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:56.219830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:56.219830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:56.219830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:56.219830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:56.219830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:56.219830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:56.225873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 335/672] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:56.225873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.231717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.232407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.232761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.233139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.233941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:56.234255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.235680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.249156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 338/673] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:56.249945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.269268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:56.337611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0f1b6-3880-4882-841f-ba16ac637a57 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.355451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1ef9e-8860-4eba-a335-6375745514ba None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.370816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.371568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.371938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.372397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.378951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.380390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.383360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:56.392017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.392622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.392897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.393199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.393889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:56.394193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.395602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:56.396008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:56.396982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:56.398048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:56.398048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:56.398048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:56.398048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:56.398048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:56.398048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:56.398048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:56.398048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:56.398486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 336/674] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:56.399608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.415714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 339/675] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:56.415714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.493641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:56.507431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f44d9-3c90-4d93-9289-adbca48eddf1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.515593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74faf0bc-d384-49b5-91d4-4ef831cec2ea None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.542461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.543073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.543346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.543617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.544299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:56.544599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.546046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.549990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.550593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.550867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.551133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.551879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=df27a2ce80b94237bd2b9ecfe35109cb) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:56.552184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.554114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.558733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 337/676] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:56.569189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 340/677] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/role_assignments?user.id=df27a2ce80b94237bd2b9ecfe35109cb&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:56.569189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.569447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.630239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:56.684207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a835e3-eeee-4a3b-98ac-8716a5b5a66e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.695238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7574d-a702-44ac-be16-d7911ee1a03d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.732027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.733171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.733723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.734000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.734708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:56.735014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.736462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.737955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.738801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.739192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.739630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.753344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.753634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 338/678] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:56.754357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.755323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.760364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:56.779918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:56.780899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:56.781839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:56.781839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:56.781839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:56.781839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 341/679] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:56.782075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.895081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0823ca0-a621-460a-9ad1-3c4863d9b291 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.912596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:56.934788 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.935694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.935924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.936293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.937148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=029447e782484e8f8a704b683ede1819) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:56.937559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.939297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905d8bc-1583-48e1-8a25-b736e2868430 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:56.939532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.953156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 339/680] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/role_assignments?user.id=029447e782484e8f8a704b683ede1819&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:56.955326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:56.983576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc61d4-e702-44db-90aa-4a26ea0d2da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:56.984198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc61d4-e702-44db-90aa-4a26ea0d2da9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.984519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc61d4-e702-44db-90aa-4a26ea0d2da9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.985187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc61d4-e702-44db-90aa-4a26ea0d2da9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:56.985945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc61d4-e702-44db-90aa-4a26ea0d2da9 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=341277e7e3144e5ea93fa657916821e1) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:56.986252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc61d4-e702-44db-90aa-4a26ea0d2da9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:56.987724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc61d4-e702-44db-90aa-4a26ea0d2da9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:56.995783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 342/681] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/role_assignments?user.id=341277e7e3144e5ea93fa657916821e1&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:56.997042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5d482-7056-4478-8229-bf763e093eb8 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:56.997348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5d482-7056-4478-8229-bf763e093eb8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:56.997609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5d482-7056-4478-8229-bf763e093eb8 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:57.003570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 343/682] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:57.005185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fce20-245b-40ea-ac49-1449378f2d0a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:57.005827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fce20-245b-40ea-ac49-1449378f2d0a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:57.006190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fce20-245b-40ea-ac49-1449378f2d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:57.031581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ef3fce20-245b-40ea-ac49-1449378f2d0a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:57.033384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ef3fce20-245b-40ea-ac49-1449378f2d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:57.129911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fefee-cefe-49a1-b982-f2cdca6a1170 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:57.134469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:57.162966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 344/683] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:57.203577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:57.204459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:57.204887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:57.205327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:57.206148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:57.206611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:57.208133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:57.221840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 340/684] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:57.227717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fce20-245b-40ea-ac49-1449378f2d0a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:57.336806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fce20-245b-40ea-ac49-1449378f2d0a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:57.376199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:57.376844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:57.377148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:57.377427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:57.386367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:57.387805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:57.401615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:57.402590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:57.403515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 345/685] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:35:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:57.409006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:57.457272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:57.480649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:57.521818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3c712-e7f5-4b90-aa43-4a38db6c7a23 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:57.543970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:57.559210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:57.559819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:57.560098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:57.560510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:57.561262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:57.561743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:57.563463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:57.583656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 341/686] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 01:35:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:57.589253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:57.600809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b4a30-6f74-4c0c-ba5e-7ccf43904195 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:57.644493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f260a-0b66-44e7-9fae-5a1d3f7ef79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:57.645167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f260a-0b66-44e7-9fae-5a1d3f7ef79c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:57.647321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f260a-0b66-44e7-9fae-5a1d3f7ef79c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:57.648968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f260a-0b66-44e7-9fae-5a1d3f7ef79c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:57.673240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f260a-0b66-44e7-9fae-5a1d3f7ef79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e63f260a-0b66-44e7-9fae-5a1d3f7ef79c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:35:57.686663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:57.687696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:35:57.688527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:57.689229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 346/687] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:57.696176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f260a-0b66-44e7-9fae-5a1d3f7ef79c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:57.728377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:57.733380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92a6b6-d12c-4200-b487-a36c8b441b1c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:57.838307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936780d-9a03-4674-97c6-f815c94f6f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:57.838918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936780d-9a03-4674-97c6-f815c94f6f9a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:57.839193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936780d-9a03-4674-97c6-f815c94f6f9a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:57.839458 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2936780d-9a03-4674-97c6-f815c94f6f9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:57.841455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f260a-0b66-44e7-9fae-5a1d3f7ef79c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:57.852002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:57.877412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:57.878411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:57.878694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:57.879100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:57.879767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:57.880070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:57.881724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:57.896528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 347/688] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:57.897332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:58.009101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d987ac-6eff-4786-94dc-81840ea30e95 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:58.048169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:58.048806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:58.049172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:58.049450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:58.062263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:58.063712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:58.077879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936780d-9a03-4674-97c6-f815c94f6f9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:58.079332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2936780d-9a03-4674-97c6-f815c94f6f9a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:58.079584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:58.080488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:58.081420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:58.081420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:58.081420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:58.085336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 348/689] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:58.087231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:58.115550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:58.202712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef71ba3-dcdb-455c-a02e-a6de626aab99 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:58.241502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:58.242244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:58.242571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:58.242850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:58.243488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:58.243793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:58.245175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:58.259614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 349/690] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:58.265420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:58.304574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:58.313889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 342/691] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:35:58.315943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936780d-9a03-4674-97c6-f815c94f6f9a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:58.428726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:58.457892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dffb5-3f0d-4efe-a473-eaa17c02fb47 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:58.483336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2936780d-9a03-4674-97c6-f815c94f6f9a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:58.514134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:58.514692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:58.514966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:58.515230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:58.525011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:58.526462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:35:58.538261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:58.539188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:58.540676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 350/692] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:35:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:58.547421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:58.548074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:58.548373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:58.548566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:58.548839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:58.576079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:58.577586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:58.606289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:58.607273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:58.608144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:58.608144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:58.608144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:58.611770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 343/693] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:58.612846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:58.661599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e940f-0746-4c82-85bc-3ea08be04023 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:58.698523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:58.699518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:58.700009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:58.700488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:58.701457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:58.701967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:58.703595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:58.709368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:58.724379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 351/694] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:35:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:58.737601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:58.744132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:58.792250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1830b-38e3-47be-8b2f-43681bea46a5 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:58.858313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:58.859318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:58.859824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:58.860320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:58.861198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:58.861711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:58.863281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:58.888263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 344/695] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:35:58.898291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:58.898835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5763411-097f-41e6-8319-39bb7a87aeca None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:58.938001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f92646-491d-4381-9f15-87bf319c5ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:58.939606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f92646-491d-4381-9f15-87bf319c5ed8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:58.939890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f92646-491d-4381-9f15-87bf319c5ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:58.940160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f92646-491d-4381-9f15-87bf319c5ed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.041319 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f92646-491d-4381-9f15-87bf319c5ed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:59.042846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f92646-491d-4381-9f15-87bf319c5ed8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:59.045237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:59.142172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864adb2e-ff04-41ea-ba96-81883afc0182 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:59.169458 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:59.180526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c19c0b-3e2e-4271-88a7-d633e3f0aa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:59.181479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c19c0b-3e2e-4271-88a7-d633e3f0aa3d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.181985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c19c0b-3e2e-4271-88a7-d633e3f0aa3d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.182453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c19c0b-3e2e-4271-88a7-d633e3f0aa3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.183632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 352/696] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 01:35:59.184954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090577-35fc-498a-8d6e-b04bc288b3eb None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.185274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090577-35fc-498a-8d6e-b04bc288b3eb None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.185541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090577-35fc-498a-8d6e-b04bc288b3eb None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.187059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 353/697] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:59.188649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08090577-35fc-498a-8d6e-b04bc288b3eb None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:59.191175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c19c0b-3e2e-4271-88a7-d633e3f0aa3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:59.193006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c19c0b-3e2e-4271-88a7-d633e3f0aa3d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-51c19c0b-3e2e-4271-88a7-d633e3f0aa3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:35:59.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:59.206628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:59.207488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:35:59.207488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:35:59.207488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:35:59.207488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:35:59.207488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:35:59.207488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:35:59.207488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:35:59.207488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:35:59.210015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 345/698] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:35:59.211458 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113108-c02c-4cda-8783-c721589f11d9 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.212032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113108-c02c-4cda-8783-c721589f11d9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.212545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c113108-c02c-4cda-8783-c721589f11d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.238326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8c113108-c02c-4cda-8783-c721589f11d9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:59.239514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8c113108-c02c-4cda-8783-c721589f11d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:59.315068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08090577-35fc-498a-8d6e-b04bc288b3eb None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:59.317249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:59.324820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 346/699] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:59.325925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113108-c02c-4cda-8783-c721589f11d9 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:59.373217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:59.373845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.374120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.376491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.405934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:59.407401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:35:59.439143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:59.440170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:35:59.441092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:35:59.441092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:35:59.441092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:59.441092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 354/700] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:59.446408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:59.506077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c113108-c02c-4cda-8783-c721589f11d9 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:59.511677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:59.598018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01a273-b18f-493b-aea7-7854c252d6fc None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:59.610682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9f242-2064-47b7-9dee-904ae5c2f92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:59.611310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9f242-2064-47b7-9dee-904ae5c2f92e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.611590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9f242-2064-47b7-9dee-904ae5c2f92e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.611861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9f242-2064-47b7-9dee-904ae5c2f92e None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.612562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9f242-2064-47b7-9dee-904ae5c2f92e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:59.612874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9f242-2064-47b7-9dee-904ae5c2f92e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:59.614592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9f242-2064-47b7-9dee-904ae5c2f92e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:59.636954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 347/701] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:59.638704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee865-73b2-4823-bc4e-6fca0f42e71d None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.639347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee865-73b2-4823-bc4e-6fca0f42e71d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.639823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ee865-73b2-4823-bc4e-6fca0f42e71d None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.641701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 348/702] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:59.642706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ee865-73b2-4823-bc4e-6fca0f42e71d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:59.651299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c628b-66ad-4548-8b91-604454dc8844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:59.651914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c628b-66ad-4548-8b91-604454dc8844 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.652194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c628b-66ad-4548-8b91-604454dc8844 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.652465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c628b-66ad-4548-8b91-604454dc8844 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.653148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c628b-66ad-4548-8b91-604454dc8844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:59.653456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c628b-66ad-4548-8b91-604454dc8844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:59.655120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c628b-66ad-4548-8b91-604454dc8844 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:59.667747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 355/703] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:59.669025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c83b-e32f-4eef-ba79-94f0f46748b0 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.669339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c83b-e32f-4eef-ba79-94f0f46748b0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.669610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9c83b-e32f-4eef-ba79-94f0f46748b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.688839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-43e9c83b-e32f-4eef-ba79-94f0f46748b0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:35:59.690598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-43e9c83b-e32f-4eef-ba79-94f0f46748b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:35:59.703567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:59.759985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ee865-73b2-4823-bc4e-6fca0f42e71d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:59.773617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 356/704] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:35:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:35:59.774358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9c83b-e32f-4eef-ba79-94f0f46748b0 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:59.807357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:59.808879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.810269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.810747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.811683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=e0d08965bc9c4ca29812d304bfb4ab4f) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:35:59.812204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:59.813848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:59.825664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 349/705] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:35:59 2022] GET /identity/v3/role_assignments?user.id=e0d08965bc9c4ca29812d304bfb4ab4f&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:35:59.825664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:59.889128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9c83b-e32f-4eef-ba79-94f0f46748b0 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:35:59.889670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:59.937811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85c296-f050-4494-910d-8208008808db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:35:59.938462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c296-f050-4494-910d-8208008808db None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.938739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c296-f050-4494-910d-8208008808db None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.939012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c296-f050-4494-910d-8208008808db None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.939609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85c296-f050-4494-910d-8208008808db None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:35:59.941007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85c296-f050-4494-910d-8208008808db None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-de85c296-f050-4494-910d-8208008808db None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:35:59.953502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:35:59.954582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:35:59.956057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 357/706] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 9 01:35:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:35:59.956887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42483725-4bc5-4914-9d48-a281fa89e1f0 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:35:59.956887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42483725-4bc5-4914-9d48-a281fa89e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:35:59.956887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42483725-4bc5-4914-9d48-a281fa89e1f0 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:35:59.959094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 358/707] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:35:59.960212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42483725-4bc5-4914-9d48-a281fa89e1f0 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:35:59.973233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:35:59.984904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f531049-6da7-400d-ae6d-bc66f958369a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.035751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dee65a-6b63-4e94-bf2c-b10dd374efb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.036394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee65a-6b63-4e94-bf2c-b10dd374efb6 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.037091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee65a-6b63-4e94-bf2c-b10dd374efb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.037091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee65a-6b63-4e94-bf2c-b10dd374efb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.046942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dee65a-6b63-4e94-bf2c-b10dd374efb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.048480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dee65a-6b63-4e94-bf2c-b10dd374efb6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-42dee65a-6b63-4e94-bf2c-b10dd374efb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:00.061453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:00.062462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:00.063464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 350/708] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:00.063983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984da2c-fcd2-4ab1-8880-7938edcfe968 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.064254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984da2c-fcd2-4ab1-8880-7938edcfe968 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.064519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984da2c-fcd2-4ab1-8880-7938edcfe968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.085616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42483725-4bc5-4914-9d48-a281fa89e1f0 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.089391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f984da2c-fcd2-4ab1-8880-7938edcfe968 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:00.090582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f984da2c-fcd2-4ab1-8880-7938edcfe968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:00.127303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.127912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.128185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.128453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.129184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:00.129435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.131078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.135492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 359/709] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 01:35:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:00.136272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.169607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 351/710] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:36:00.170725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984da2c-fcd2-4ab1-8880-7938edcfe968 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.195328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:00.221769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:00.255052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17582d1a-2ff9-4a04-ad79-0ddf53c814ca None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.298747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.299268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984da2c-fcd2-4ab1-8880-7938edcfe968 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.299972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.300453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.301205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] PATH_INFO: `/v3/services/network` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.302862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.304423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:00.317217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:00.318185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:00.319048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:00.323405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 360/711] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:00 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:00.324431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.338938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.339793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.340084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.340458 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.341289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:00.341906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.343341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.360674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 352/712] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:00.360674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.439280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:00.463135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd17ad6-b5a3-4bb4-8880-12f5e5cfae4e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.465570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815675a-4f8c-454f-8316-4ce7c35fe0ee None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.498464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.499065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.499340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.499596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.499940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.500209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:00.500502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.500612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.500794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.501073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.501696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] RBAC: Adding query filter params (name=network) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:00.501885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.501995 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.503306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.505645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:00.512274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 353/713] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:00.512274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.512526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 361/714] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 01:36:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:00.512526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.623417 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c73d75-f4bd-4660-ab36-fb449c18a549 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.637876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236efa20-e336-45cb-b5f4-fd679a8058bd None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.662639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.663694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.664181 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.665242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.676432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.677339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.677339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.677650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.678360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:00.678666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.680206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.680718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.682628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.693614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 354/715] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:00 2022] GET /identity/v3/role_assignments?user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=b131b61299154544a627315649090a49 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:00.694537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:00.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:00.702347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:00.703235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:00.703235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:00.703235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:00.704854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:00.706042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 362/716] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:00.712828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.776398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:00.835802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b495f-1945-4912-b213-46278ec92e45 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.853334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bb1ac-f98b-47f1-91a1-9bd3bb1a52cd None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:00.878123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.879101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.879613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.880096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.881831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:00.882367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.884710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.904061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 363/717] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:00.910365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:00.913046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa401f-5566-483e-808a-9dfc5da9b0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:00.913862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa401f-5566-483e-808a-9dfc5da9b0d5 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:00.914206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa401f-5566-483e-808a-9dfc5da9b0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:00.915270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa401f-5566-483e-808a-9dfc5da9b0d5 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:00.918384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa401f-5566-483e-808a-9dfc5da9b0d5 None admin] RBAC: Adding query filter params (type=network) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:00.918384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa401f-5566-483e-808a-9dfc5da9b0d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:00.918384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa401f-5566-483e-808a-9dfc5da9b0d5 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:00.924422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 355/718] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:01.063885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ee831-e99d-4b2a-b58d-769243c1c5c3 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:01.080456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:01.110368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358c68-c330-41bd-a833-81e99f13d736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:01.111348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358c68-c330-41bd-a833-81e99f13d736 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:01.111848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358c68-c330-41bd-a833-81e99f13d736 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:01.112120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358c68-c330-41bd-a833-81e99f13d736 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:01.112776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358c68-c330-41bd-a833-81e99f13d736 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:01.114238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358c68-c330-41bd-a833-81e99f13d736 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:01.144684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 364/719] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:36:00 2022] POST /identity/v3/users => generated 254 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:02.237002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2336f9-e5e4-4f23-aaf1-ec703e4d65bb None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:02.237978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2336f9-e5e4-4f23-aaf1-ec703e4d65bb None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:02.238471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2336f9-e5e4-4f23-aaf1-ec703e4d65bb None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:02.240308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 356/720] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:02.249439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fedd6-72b7-4203-ad3f-b0d05b456ede None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:02.249439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fedd6-72b7-4203-ad3f-b0d05b456ede None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:02.249439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fedd6-72b7-4203-ad3f-b0d05b456ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:02.269576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a02fedd6-72b7-4203-ad3f-b0d05b456ede None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:02.271031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a02fedd6-72b7-4203-ad3f-b0d05b456ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:02.363630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 365/721] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:36:02.775711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2336f9-e5e4-4f23-aaf1-ec703e4d65bb None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:02.854850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:02.910886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2336f9-e5e4-4f23-aaf1-ec703e4d65bb None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:02.977936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af1175-e830-4c05-9d25-027f40317f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:02.978572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af1175-e830-4c05-9d25-027f40317f0b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:02.978852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af1175-e830-4c05-9d25-027f40317f0b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:02.979122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af1175-e830-4c05-9d25-027f40317f0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:02.995884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af1175-e830-4c05-9d25-027f40317f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-e2af1175-e830-4c05-9d25-027f40317f0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:03.000865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:03.001798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:03.002617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:03.006908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 357/722] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:03.008113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fedd6-72b7-4203-ad3f-b0d05b456ede None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:03.043279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa196c-639f-44c2-875b-dd5a979ff096 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.043556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa196c-639f-44c2-875b-dd5a979ff096 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.043750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa196c-639f-44c2-875b-dd5a979ff096 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.045595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 358/723] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:03.050855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83a3e-701e-40c9-8bc9-e858176544b6 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.051140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83a3e-701e-40c9-8bc9-e858176544b6 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.051407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83a3e-701e-40c9-8bc9-e858176544b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.082222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-55f83a3e-701e-40c9-8bc9-e858176544b6 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:03.083442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-55f83a3e-701e-40c9-8bc9-e858176544b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:03.140609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fedd6-72b7-4203-ad3f-b0d05b456ede None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:03.142268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:03.178966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:03.179599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.179911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.180184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.180877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:03.181254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:03.185122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:03.192973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 366/724] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:03.198495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:03.257542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 359/725] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 01:36:03.311795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d1fe9-9413-41f7-8ae1-479b1c750fa4 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:03.332666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bba147-eb6d-478b-b543-1e9190673c8a None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.333091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bba147-eb6d-478b-b543-1e9190673c8a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.333515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bba147-eb6d-478b-b543-1e9190673c8a None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.335884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 360/726] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:03.344047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21613e01-99ca-4ea7-9532-205b9c07a742 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.344857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21613e01-99ca-4ea7-9532-205b9c07a742 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.345480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21613e01-99ca-4ea7-9532-205b9c07a742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.359106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:03.359758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.360037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.360309 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.370367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-21613e01-99ca-4ea7-9532-205b9c07a742 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:03.371770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-21613e01-99ca-4ea7-9532-205b9c07a742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:03.384065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:03.385649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:03.415112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:03.417470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:03.419607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:03.419607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:03.419607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:03.419607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 367/727] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:03.426208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:03.444117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:03.460913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 361/728] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:36:03.465863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:03.588122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21613e01-99ca-4ea7-9532-205b9c07a742 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:03.593535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50ac77-b1d3-4fb4-b5b1-19a517a00e1f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:03.657142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:03.657781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.658110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.658385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.659045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:03.659345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:03.660748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:03.688757 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 368/729] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:03.696512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:03.732206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21613e01-99ca-4ea7-9532-205b9c07a742 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:03.776057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a40a-b9a9-4fe6-a427-9e5a7527ada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:03.776743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a40a-b9a9-4fe6-a427-9e5a7527ada7 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.777088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a40a-b9a9-4fe6-a427-9e5a7527ada7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.777588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504a40a-b9a9-4fe6-a427-9e5a7527ada7 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.778316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504a40a-b9a9-4fe6-a427-9e5a7527ada7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:03.779742 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504a40a-b9a9-4fe6-a427-9e5a7527ada7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:03.806745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 362/730] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:36:03 2022] POST /identity/v3/services => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:03.806745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a40a-b9a9-4fe6-a427-9e5a7527ada7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:03.834216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:03.870375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debeb902-a8aa-41d5-a672-70456e0f5c46 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:03.881482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:03.925340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34a903-3d23-4431-af91-187ff4361275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:03.926311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34a903-3d23-4431-af91-187ff4361275 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.926768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34a903-3d23-4431-af91-187ff4361275 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.927189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34a903-3d23-4431-af91-187ff4361275 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.937210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34a903-3d23-4431-af91-187ff4361275 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:03.939144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34a903-3d23-4431-af91-187ff4361275 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:03.939631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504a40a-b9a9-4fe6-a427-9e5a7527ada7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-fa34a903-3d23-4431-af91-187ff4361275 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:03.952943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:03.953985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:36:03.954925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:03.958013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 369/731] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:03.959711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d173b3-bb6d-482f-a7dc-f42761e16208 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.960228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d173b3-bb6d-482f-a7dc-f42761e16208 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.960708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d173b3-bb6d-482f-a7dc-f42761e16208 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.967487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 370/732] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:03.968764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d173b3-bb6d-482f-a7dc-f42761e16208 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:03.981099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f419e-3e00-479f-8522-902dab6c2356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:03.981796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f419e-3e00-479f-8522-902dab6c2356 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:03.982079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f419e-3e00-479f-8522-902dab6c2356 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:03.982348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f419e-3e00-479f-8522-902dab6c2356 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:03.996641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f419e-3e00-479f-8522-902dab6c2356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-a81f419e-3e00-479f-8522-902dab6c2356 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:04.002117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:04.003054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:04.003893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 363/733] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:04.004622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46faf92-73c6-4611-a046-7d6db144ed9a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.006752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46faf92-73c6-4611-a046-7d6db144ed9a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.006752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46faf92-73c6-4611-a046-7d6db144ed9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.027972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b46faf92-73c6-4611-a046-7d6db144ed9a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:04.029731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b46faf92-73c6-4611-a046-7d6db144ed9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:04.104623 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 364/734] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:36:04.105706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46faf92-73c6-4611-a046-7d6db144ed9a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:04.119631 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d173b3-bb6d-482f-a7dc-f42761e16208 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:04.149503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:04.157754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:04.167409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:04.168021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.168303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.168579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.169472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:04.170013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:04.172518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:04.192022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 371/735] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 01:36:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:04.197617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:04.229524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46faf92-73c6-4611-a046-7d6db144ed9a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:04.269228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:04.270721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.271215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.271643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.272459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:04.272929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:04.275004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:04.285255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 365/736] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:04.290847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:04.404385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28376aa-c7d6-47de-bcc7-04c313a69ea2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:04.412933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:04.416550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0dabdf-d17e-402f-b6d6-45dabe8c4e50 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:04.467313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecba73a-9806-413f-9f1d-0e474f8c705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:04.467940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecba73a-9806-413f-9f1d-0e474f8c705b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.468210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecba73a-9806-413f-9f1d-0e474f8c705b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.468478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecba73a-9806-413f-9f1d-0e474f8c705b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.482884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83799b-2cf5-4220-b9ac-275a58249de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:04.483492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83799b-2cf5-4220-b9ac-275a58249de9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.483770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83799b-2cf5-4220-b9ac-275a58249de9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.484035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83799b-2cf5-4220-b9ac-275a58249de9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.485708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecba73a-9806-413f-9f1d-0e474f8c705b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:04.487133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecba73a-9806-413f-9f1d-0e474f8c705b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-6ecba73a-9806-413f-9f1d-0e474f8c705b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:04.509869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:04.510861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:04.511768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:04.511768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:04.511768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:04.513702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 366/737] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:04.519666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3537-6cd9-4b2d-a562-6700887ae1bc None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.520161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3537-6cd9-4b2d-a562-6700887ae1bc None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.520644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a3537-6cd9-4b2d-a562-6700887ae1bc None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.522377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 367/738] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:04.523282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a3537-6cd9-4b2d-a562-6700887ae1bc None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:04.548152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:04.562720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83799b-2cf5-4220-b9ac-275a58249de9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:04.564343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c83799b-2cf5-4220-b9ac-275a58249de9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:04.634757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a3537-6cd9-4b2d-a562-6700887ae1bc None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:04.663088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 372/739] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:04 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 01:36:04.664151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073157c3-ec3d-4c34-89af-d63d57a18bee None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.664482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073157c3-ec3d-4c34-89af-d63d57a18bee None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.664753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073157c3-ec3d-4c34-89af-d63d57a18bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.674274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:04.674879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.675152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.675417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.676049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:04.676465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:04.677941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:04.682754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-073157c3-ec3d-4c34-89af-d63d57a18bee None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:04.683960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-073157c3-ec3d-4c34-89af-d63d57a18bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:04.694913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 368/740] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:04.694913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:04.707569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:04.761682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 373/741] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 01:36:04.762507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073157c3-ec3d-4c34-89af-d63d57a18bee None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:04.780125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:04.818813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd643b-6ce7-4215-b3c2-e769b014192e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:04.862209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b208eef-25e5-462c-aba8-edb74b65a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:04.863438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b208eef-25e5-462c-aba8-edb74b65a4f4 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.863940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b208eef-25e5-462c-aba8-edb74b65a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.864561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b208eef-25e5-462c-aba8-edb74b65a4f4 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:04.865798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b208eef-25e5-462c-aba8-edb74b65a4f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:04.868487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b208eef-25e5-462c-aba8-edb74b65a4f4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:04.886496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073157c3-ec3d-4c34-89af-d63d57a18bee None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:04.897722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 369/742] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:36:04 2022] POST /identity/v3/services => generated 240 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:04.899111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b208eef-25e5-462c-aba8-edb74b65a4f4 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:04.980349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:04.983542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:04.984036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:04.985182 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.014145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.015800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.037658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:05.038582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:05.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:05.041246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:05.041246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:05.041246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:05.041246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 374/743] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:05.041246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.081853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b208eef-25e5-462c-aba8-edb74b65a4f4 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.127631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.128259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.128533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.128800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.134630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:05.140312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.142388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:05.155755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:05.156754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:05.157715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 370/744] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:36:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:05.158316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.191016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d8498-4331-4699-9231-df92ee2dc0f1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.233462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf788e2b-c496-4f4f-aac1-1217abdbd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.234026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf788e2b-c496-4f4f-aac1-1217abdbd5f2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.234562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf788e2b-c496-4f4f-aac1-1217abdbd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.234847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf788e2b-c496-4f4f-aac1-1217abdbd5f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.248632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf788e2b-c496-4f4f-aac1-1217abdbd5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-cf788e2b-c496-4f4f-aac1-1217abdbd5f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:05.256643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:05.257598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:05.258738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 375/745] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:05.259419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf788e2b-c496-4f4f-aac1-1217abdbd5f2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.290054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b02b7-2625-4dca-8695-c26ae3caf266 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.329490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:05.337153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.338011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.338598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.339387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.340507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:05.340917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.342866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.365154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 371/746] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:05.368001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.394342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf788e2b-c496-4f4f-aac1-1217abdbd5f2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.422458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:05.435822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.436637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.437289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.437783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.438762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:05.439271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.441127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.460524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 376/747] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:05.462226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.494422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185f972-c79f-4b1e-929a-04d7eca8b49e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.534805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.535492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.535773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.536325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.536996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:05.537355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.539528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.555987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 372/748] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:05.555987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.593126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7195e-442d-415f-8e2d-becc152ec5af None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.613682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:05.636670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.637704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.638212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.638685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.646859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.649209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:05.661820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:05.662747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:05.663625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:05.663625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:05.663625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:05.663625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:05.663625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:05.663625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:05.663625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:05.663625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:05.666207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 377/749] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:05.667368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.675047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abefa24-81a5-4115-8613-c65e871004eb None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.714271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.715487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.715969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.716464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.719718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:05.730463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.732483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:05.748572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:05.749584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:05.750484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:05.750484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:05.750484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:05.752657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 373/750] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:05.753708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.788854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cdf78-f22f-4e1e-b0df-9bfc9ae4ae59 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.827408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.828421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.828852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.829292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.841400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.843395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:05.861361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:05.862570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:05.863555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:05.863555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:05.863555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:05.869097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d784bd2-369c-41bf-887a-e437751e03a5 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:05.869294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 378/751] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:05.870441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.876540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:05.906889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:05.907501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:05.907778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:05.908052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:05.908738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:05.909042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:05.910498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:05.923699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 374/752] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:05.924792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:05.981374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:05.990884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79c619-8068-40fc-86b1-739964b75abb None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:06.032888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:06.033791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.034221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.034660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.035110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926796a9-ae77-4bbf-a39f-4511e1625294 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:06.035608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:06.036131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:06.037865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:06.056384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 379/753] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:06.057682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:06.078482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:06.079097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.079373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.079647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.080457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:06.080858 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:06.083275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:06.100696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 375/754] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:06.101686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:06.130166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:06.176102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2cbda-7404-45c8-b22f-e90c972da90a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:06.207997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069996e5-8a05-40ac-b155-0eb600a379ac None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:06.214285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:06.215040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.215323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.215591 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.216308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=e0d08965bc9c4ca29812d304bfb4ab4f) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:06.216622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:06.218059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:06.223809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 380/755] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:06 2022] GET /identity/v3/role_assignments?user.id=e0d08965bc9c4ca29812d304bfb4ab4f&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=b131b61299154544a627315649090a49 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:06.224385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:06.254127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:06.257329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa7517-4f4d-4fcf-ad51-741dab6d2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:06.258075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7517-4f4d-4fcf-ad51-741dab6d2ae6 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.258350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7517-4f4d-4fcf-ad51-741dab6d2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.258672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa7517-4f4d-4fcf-ad51-741dab6d2ae6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.266556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa7517-4f4d-4fcf-ad51-741dab6d2ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:06.268031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa7517-4f4d-4fcf-ad51-741dab6d2ae6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-eafa7517-4f4d-4fcf-ad51-741dab6d2ae6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:06.281317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:06.282283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:06.283875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 376/756] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:06.284367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c290c89-91d7-4c8b-b45b-ed19bce4d2c7 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.284367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c290c89-91d7-4c8b-b45b-ed19bce4d2c7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.284706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c290c89-91d7-4c8b-b45b-ed19bce4d2c7 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.290383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 377/757] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:06.290383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c290c89-91d7-4c8b-b45b-ed19bce4d2c7 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:06.380981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d7a9-98ac-41c3-90eb-e65dd2f24099 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:06.389888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:06.409893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c290c89-91d7-4c8b-b45b-ed19bce4d2c7 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:06.428372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38d19a-ddba-4e1f-bf22-27ab629fc43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:06.429269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38d19a-ddba-4e1f-bf22-27ab629fc43b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.429614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38d19a-ddba-4e1f-bf22-27ab629fc43b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.430157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38d19a-ddba-4e1f-bf22-27ab629fc43b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.440431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38d19a-ddba-4e1f-bf22-27ab629fc43b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:06.442499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38d19a-ddba-4e1f-bf22-27ab629fc43b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:06.450145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:06.450836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.451105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.451292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.452044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:06.452409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:06.453926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-fd38d19a-ddba-4e1f-bf22-27ab629fc43b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:06.457765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:06.458659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:06.459589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:06.462661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 381/758] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:36:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:06.464835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359c94-fda7-4fe4-8277-f5b3fcb80bec None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.465358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359c94-fda7-4fe4-8277-f5b3fcb80bec None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.465836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94359c94-fda7-4fe4-8277-f5b3fcb80bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.466723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 378/759] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:06.474077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:06.487075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-94359c94-fda7-4fe4-8277-f5b3fcb80bec None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:06.488765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-94359c94-fda7-4fe4-8277-f5b3fcb80bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:06.582676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 382/760] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:06.582676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94359c94-fda7-4fe4-8277-f5b3fcb80bec None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:06.588669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d08f-e9bf-4001-84e1-85793d8f52be None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:06.601400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:06.657466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f3268-5049-48bb-a763-3fcf2e5bdac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:06.658136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3268-5049-48bb-a763-3fcf2e5bdac8 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.658412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3268-5049-48bb-a763-3fcf2e5bdac8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.658680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f3268-5049-48bb-a763-3fcf2e5bdac8 None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/5a63d93adb4e4044ae79fb5aa7d8f5e3/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.662223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f3268-5049-48bb-a763-3fcf2e5bdac8 None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=5a63d93adb4e4044ae79fb5aa7d8f5e3, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:06.663948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13f3268-5049-48bb-a763-3fcf2e5bdac8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:06.688083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 379/761] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:36:06 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/5a63d93adb4e4044ae79fb5aa7d8f5e3/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:36:06.751707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94359c94-fda7-4fe4-8277-f5b3fcb80bec None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:06.802121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:06.803138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:06.803616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:06.804079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:06.805707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:06.806271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:06.807971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:06.834330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 383/762] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 01:36:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:06.839643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f3268-5049-48bb-a763-3fcf2e5bdac8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:06.854364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:06.993887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:07.024300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13f3268-5049-48bb-a763-3fcf2e5bdac8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:07.081864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:07.095558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3049c-f1c6-4582-a655-3e20e169c5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:07.096109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3049c-f1c6-4582-a655-3e20e169c5ac None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.096388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3049c-f1c6-4582-a655-3e20e169c5ac None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.096661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3049c-f1c6-4582-a655-3e20e169c5ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.120230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4af2e8-06c6-4644-95ad-ed8d7dd5f7a5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:07.162616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:07.163611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.164223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.164701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.165658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:07.167206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:07.176423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:07.177427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:36:07.178243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:07.180974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 384/763] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 9 01:36:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:07.186725 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:07.261970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3049c-f1c6-4582-a655-3e20e169c5ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:07.263578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3049c-f1c6-4582-a655-3e20e169c5ac None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:07.271545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:07.302822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77120ff1-be49-4b45-9ca6-6c8176b8e42d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:07.337620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:07.344738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186d0d0-4984-4b21-88f3-54e12603af5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:07.346096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d0d0-4984-4b21-88f3-54e12603af5c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.346411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d0d0-4984-4b21-88f3-54e12603af5c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.346779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d0d0-4984-4b21-88f3-54e12603af5c None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.347497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d0d0-4984-4b21-88f3-54e12603af5c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:07.347894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d0d0-4984-4b21-88f3-54e12603af5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:07.350223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186d0d0-4984-4b21-88f3-54e12603af5c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:07.355832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 385/764] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 01:36:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:07.357816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2671fd-b138-40d7-a5d7-2539906f24bc None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.358199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2671fd-b138-40d7-a5d7-2539906f24bc None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.358579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2671fd-b138-40d7-a5d7-2539906f24bc None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.364121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 386/765] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:07.364121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2671fd-b138-40d7-a5d7-2539906f24bc None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:07.379694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 380/766] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:07.381964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b7c93-ad83-4f28-bf35-2c2bc9b45f9d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.382396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b7c93-ad83-4f28-bf35-2c2bc9b45f9d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.383440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b7c93-ad83-4f28-bf35-2c2bc9b45f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.403370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1b9b7c93-ad83-4f28-bf35-2c2bc9b45f9d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:07.404872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1b9b7c93-ad83-4f28-bf35-2c2bc9b45f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:07.481119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2671fd-b138-40d7-a5d7-2539906f24bc None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:07.481645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 381/767] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:07.482437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b7c93-ad83-4f28-bf35-2c2bc9b45f9d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:07.515444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:07.516057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.516336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.516606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.517289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:07.517564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:07.519609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:07.524975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 387/768] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 01:36:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:07.531130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:07.558060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:07.603974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:07.657175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc06d16-396c-436e-ba5f-daf70212a563 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:07.707086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b7c93-ad83-4f28-bf35-2c2bc9b45f9d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:07.709208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:07.709824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.711558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.711966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.712663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] RBAC: Adding query filter params (interface=public, service_id=2fae97cdcaee47188a2a90abe3ab886e, region_id=RegionOne) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:07.712971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:07.714507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:07.727630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 388/769] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 01:36:07 2022] GET /identity/v3/endpoints?service_id=2fae97cdcaee47188a2a90abe3ab886e&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:07.728832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:07.800030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:07.800681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.800957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.801265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.814827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:07.816634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:07.836737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:07.837848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:07.838720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:07.838720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:07.838720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:07.838901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66ed05-bc0b-4282-8e14-37773a939fbd None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:07.841159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 382/770] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:07.842045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:07.873543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:07.878732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774d5c7-90ff-4573-9327-c8ef41b5571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:07.879454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774d5c7-90ff-4573-9327-c8ef41b5571e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:07.879804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774d5c7-90ff-4573-9327-c8ef41b5571e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:07.880167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774d5c7-90ff-4573-9327-c8ef41b5571e None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:07.881152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774d5c7-90ff-4573-9327-c8ef41b5571e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:07.882669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774d5c7-90ff-4573-9327-c8ef41b5571e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:07.888702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 389/771] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:36:07 2022] GET /identity/v3/services => generated 772 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:07.902312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774d5c7-90ff-4573-9327-c8ef41b5571e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:08.009738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:08.022558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa093c4a-646f-4245-8c15-3fb462505ea8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.047422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774d5c7-90ff-4573-9327-c8ef41b5571e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.058372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.059417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.059897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.060375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] PATH_INFO: `/v3/services/network` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.061187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.062779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:08.075301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:08.076348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:08.077212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:08.080000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 383/772] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:07 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:08.091739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:08.120881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.121985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.122516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.123013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.123932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:08.124449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.126085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.149486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 390/773] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:08.150925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:08.211840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:08.237879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a5404-ce72-428f-9f47-7c7668f102bd None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.281710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.282820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.283313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.283784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.285402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] RBAC: Adding query filter params (name=network) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:08.286442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.287997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.288295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa9c22-15e5-4d0a-9864-3b91ce7003be None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.294269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 384/774] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:08.300112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:08.330628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.331255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.331526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.331788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.336416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:08.347685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.349162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:08.381706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:08.382625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:08.383562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 391/775] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:08.394001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:08.446194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b38352-b95a-4ce8-9e62-f7e80d4809a0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.483165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.483773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.484050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.484318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.484937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] RBAC: Adding query filter params (type=network) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:08.485263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.486751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.492848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 385/776] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:08.502019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:08.603572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a8ec0-baab-4702-840c-fad6c4ab86f5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.618481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:08.629401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:08.658694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40517a6d-1a74-484e-ae39-24016bf63060 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.662600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244e3d7-b074-4746-8b8d-61019ae3c7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.663305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e3d7-b074-4746-8b8d-61019ae3c7e2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.663649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e3d7-b074-4746-8b8d-61019ae3c7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.663990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e3d7-b074-4746-8b8d-61019ae3c7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.664821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244e3d7-b074-4746-8b8d-61019ae3c7e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:08.665245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244e3d7-b074-4746-8b8d-61019ae3c7e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.667203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244e3d7-b074-4746-8b8d-61019ae3c7e2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.681992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 392/777] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:08.683332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fced88-51aa-4394-807a-6cde12a40742 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.683686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fced88-51aa-4394-807a-6cde12a40742 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.684024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fced88-51aa-4394-807a-6cde12a40742 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.685600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 393/778] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:08.690634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c56f9-dc31-4008-b087-1c180d9ddb31 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.690917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c56f9-dc31-4008-b087-1c180d9ddb31 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.691190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c56f9-dc31-4008-b087-1c180d9ddb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.702217 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.702836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.703109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.703375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.704119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] RBAC: Adding query filter params (interface=public, service_id=927502024cb14adc9321b34bec60c92e, region_id=RegionOne) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:08.704425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.706249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.712246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-720c56f9-dc31-4008-b087-1c180d9ddb31 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:08.713788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-720c56f9-dc31-4008-b087-1c180d9ddb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:08.715196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 386/779] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 01:36:08 2022] GET /identity/v3/endpoints?service_id=927502024cb14adc9321b34bec60c92e&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:08.716316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:08.791792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 394/780] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:08.793307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c56f9-dc31-4008-b087-1c180d9ddb31 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:08.828601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cfac6-7bf7-46b0-92f5-1959f0d5856f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.865292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.865897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.866170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.866435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.867464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=9cfe3dd0af154707bdaae5b52cbd6539) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:08.867772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.869330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.881213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 387/781] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:08 2022] GET /identity/v3/role_assignments?user.id=9cfe3dd0af154707bdaae5b52cbd6539&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:08.912751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:08.923346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c56f9-dc31-4008-b087-1c180d9ddb31 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:08.980684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e2a3e-abbc-487e-ade4-8e907c149cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:08.981355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e2a3e-abbc-487e-ade4-8e907c149cc1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:08.981643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e2a3e-abbc-487e-ade4-8e907c149cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:08.981918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e2a3e-abbc-487e-ade4-8e907c149cc1 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:08.982539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e2a3e-abbc-487e-ade4-8e907c149cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:08.983950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e2a3e-abbc-487e-ade4-8e907c149cc1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:08.994288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 395/782] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:36:08 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:09.132517 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:09.145248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:09.256100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5fba5-1fe7-4899-93fc-722ce593b7f7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:09.295159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4a3537-c35c-4b24-afc6-a9c0c546c8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:09.295806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a3537-c35c-4b24-afc6-a9c0c546c8a2 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:09.296084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a3537-c35c-4b24-afc6-a9c0c546c8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:09.296353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4a3537-c35c-4b24-afc6-a9c0c546c8a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:09.308753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4a3537-c35c-4b24-afc6-a9c0c546c8a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-1d4a3537-c35c-4b24-afc6-a9c0c546c8a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:09.314366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:09.315387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:09.316407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 388/783] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:09.321621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e2a3e-abbc-487e-ade4-8e907c149cc1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:09.427409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e2a3e-abbc-487e-ade4-8e907c149cc1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:09.444522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a12e-a310-49bd-a174-6b8831c85473 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:09.445786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a12e-a310-49bd-a174-6b8831c85473 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:09.446310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17a12e-a310-49bd-a174-6b8831c85473 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:09.448865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 389/784] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:09.454334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af7878-10df-488c-ad6b-1efe68370153 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:09.454877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af7878-10df-488c-ad6b-1efe68370153 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:09.455349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af7878-10df-488c-ad6b-1efe68370153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:09.465990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:09.466600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:09.466873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:09.467140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:09.467797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:09.468099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:09.469487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:09.477341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-58af7878-10df-488c-ad6b-1efe68370153 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:09.479367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 396/785] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:09.479682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-58af7878-10df-488c-ad6b-1efe68370153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:09.484146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:09.515978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:09.561407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:09.575019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 390/786] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:09.607039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc3797-785e-4c39-bfb1-07ffd0f28c9c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:09.642985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:09.643609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:09.643933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:09.644209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:09.655297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:09.656923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:09.672199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:09.673165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:09.674045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:09.674045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:09.674045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:09.674045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 397/787] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:09.679372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af7878-10df-488c-ad6b-1efe68370153 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:09.824795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af7878-10df-488c-ad6b-1efe68370153 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:09.864709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:09.865688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:09.866149 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:09.866596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:09.867973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:09.868454 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:09.870025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:09.884230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 391/788] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:09.890175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:09.923540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:09.991276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:10.034957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:10.071855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9a87-8adf-476b-920e-8122a46dce77 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:10.087800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a5739b-f930-467d-9e1c-de32e029f963 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:10.143539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:10.143846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985327ef-5d3a-4351-a1c9-8a44fb44266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:10.144175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.144453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.144725 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.145441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985327ef-5d3a-4351-a1c9-8a44fb44266e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.145924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985327ef-5d3a-4351-a1c9-8a44fb44266e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.146406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985327ef-5d3a-4351-a1c9-8a44fb44266e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.152740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:10.154212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:10.157703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985327ef-5d3a-4351-a1c9-8a44fb44266e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-985327ef-5d3a-4351-a1c9-8a44fb44266e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:10.163213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:10.164669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:10.165523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:10.166221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:10.167357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:36:10.168494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:10.170417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 392/789] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:10.170700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 398/790] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:10.175978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985327ef-5d3a-4351-a1c9-8a44fb44266e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:10.182043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:10.330920 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262460f-4419-42d4-94e8-b4ae9c9c3b23 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:10.370467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:10.371539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.372278 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.372803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.373707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:10.374210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:10.375781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:10.377555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:10.387740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 399/791] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:10.393826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:10.398055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985327ef-5d3a-4351-a1c9-8a44fb44266e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:10.417513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:10.436314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:10.437336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.437826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.438389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.439315 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:10.439817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:10.441514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:10.460905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 393/792] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 01:36:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:10.466557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:10.602879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6a792-c375-4f4a-af32-35f5b5bd4421 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:10.639183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f743993-3b65-49bf-a88f-0b9a89fe3ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:10.640140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f743993-3b65-49bf-a88f-0b9a89fe3ec6 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.640638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f743993-3b65-49bf-a88f-0b9a89fe3ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.641198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f743993-3b65-49bf-a88f-0b9a89fe3ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.652669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f743993-3b65-49bf-a88f-0b9a89fe3ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:10.655097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f743993-3b65-49bf-a88f-0b9a89fe3ec6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:10.659279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b2a19-e0f0-439a-8635-920ec31d02b9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-9f743993-3b65-49bf-a88f-0b9a89fe3ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:10.672635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:10.673643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:10.674524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:10.674524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:10.674524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:10.674524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 394/793] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:10.675404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9108bc-d742-4e8a-8f21-0b789787d322 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.675691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9108bc-d742-4e8a-8f21-0b789787d322 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.675962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9108bc-d742-4e8a-8f21-0b789787d322 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.681532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 395/794] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:10.681532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9108bc-d742-4e8a-8f21-0b789787d322 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:10.706194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da41b2-f78b-4f82-a46f-85a973c6ec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:10.707440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da41b2-f78b-4f82-a46f-85a973c6ec83 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.707949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da41b2-f78b-4f82-a46f-85a973c6ec83 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.708421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da41b2-f78b-4f82-a46f-85a973c6ec83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.717461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:10.721276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da41b2-f78b-4f82-a46f-85a973c6ec83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:10.723662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da41b2-f78b-4f82-a46f-85a973c6ec83 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-f2da41b2-f78b-4f82-a46f-85a973c6ec83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:10.741634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:10.743148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:10.744025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:10.744025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:10.744025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:10.746589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 400/795] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:10.747898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b7ac4-d904-4ad4-ab98-eaa26224fe65 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.748388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b7ac4-d904-4ad4-ab98-eaa26224fe65 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.748887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923b7ac4-d904-4ad4-ab98-eaa26224fe65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.770256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-923b7ac4-d904-4ad4-ab98-eaa26224fe65 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:10.771763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-923b7ac4-d904-4ad4-ab98-eaa26224fe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:10.806327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9108bc-d742-4e8a-8f21-0b789787d322 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:10.824977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:10.861379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:10.861379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:10.861379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:10.861379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:10.861895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:10.861895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:10.861895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:10.862352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 401/796] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:10.863614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b7ac4-d904-4ad4-ab98-eaa26224fe65 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:10.880312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 396/797] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:10.886356 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.081473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:11.109203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c2967-7c51-41a5-8980-4bd2dc50a773 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:11.175884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:11.176853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.177372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.177931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.185895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923b7ac4-d904-4ad4-ab98-eaa26224fe65 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:11.191586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:11.193435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:11.205876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:11.206756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:11.207648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:11.207648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:11.207648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:11.207648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:11.207648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:11.207648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:11.207648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:11.207648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:11.210199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 397/798] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:11.211231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.225322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:11.226125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.226918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.227291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.228001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:11.228436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:11.230033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:11.248797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 402/799] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:11.248797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.297543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:11.354906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713799-3130-4bb7-93b8-37e2dfc15a43 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:11.368986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a69132-0517-448d-9ce5-4dbf47989084 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:11.395720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:11.396555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.396843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.397150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.397735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:11.399641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:11.406446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964bc6fa-b457-4cb9-aa1f-651769e4adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:11.407074 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bc6fa-b457-4cb9-aa1f-651769e4adcf None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.407354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bc6fa-b457-4cb9-aa1f-651769e4adcf None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.407625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964bc6fa-b457-4cb9-aa1f-651769e4adcf None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.408314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bc6fa-b457-4cb9-aa1f-651769e4adcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:11.408622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bc6fa-b457-4cb9-aa1f-651769e4adcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:11.409114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:11.410059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 01:36:11.410900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:11.411630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964bc6fa-b457-4cb9-aa1f-651769e4adcf None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:11.413222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 398/800] 10.222.0.15 () {56 vars in 1129 bytes} [Fri Sep 9 01:36:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:11.414628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.430209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 403/801] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:11.430209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306921af-8afe-45aa-ad86-19060bdd0f33 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.430209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306921af-8afe-45aa-ad86-19060bdd0f33 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.430209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306921af-8afe-45aa-ad86-19060bdd0f33 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.430209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 404/802] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:11.430209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306921af-8afe-45aa-ad86-19060bdd0f33 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.436251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:11.532103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c9d2-be1a-47c4-888d-849721d4d211 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:11.542016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:11.549661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306921af-8afe-45aa-ad86-19060bdd0f33 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:11.573335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83762a-deef-4a9f-94bb-f27da61f0e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:11.574063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83762a-deef-4a9f-94bb-f27da61f0e69 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.574402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83762a-deef-4a9f-94bb-f27da61f0e69 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.574740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83762a-deef-4a9f-94bb-f27da61f0e69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.583040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83762a-deef-4a9f-94bb-f27da61f0e69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:11.584430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:11.584606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83762a-deef-4a9f-94bb-f27da61f0e69 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:11.585043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.585351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.585622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.586770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:11.587068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:11.588406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:11.592958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 405/803] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 01:36:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:11.593746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-1c83762a-deef-4a9f-94bb-f27da61f0e69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:11.597462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:11.598522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:11.599626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:11.601474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 399/804] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:36:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:11.603246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a23860-7f7b-4ec1-bcda-9e6b7bb132b9 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.603528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a23860-7f7b-4ec1-bcda-9e6b7bb132b9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.603799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a23860-7f7b-4ec1-bcda-9e6b7bb132b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.621567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-17a23860-7f7b-4ec1-bcda-9e6b7bb132b9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:11.622743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-17a23860-7f7b-4ec1-bcda-9e6b7bb132b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:11.692105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 400/805] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:11.693226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a23860-7f7b-4ec1-bcda-9e6b7bb132b9 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.697419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:11.733833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b53e-18af-4b23-a696-8ae85355a73a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:11.779749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f7bbf-c36c-491e-a95b-89442dd97e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:11.780861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7bbf-c36c-491e-a95b-89442dd97e43 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.781448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7bbf-c36c-491e-a95b-89442dd97e43 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.782682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f7bbf-c36c-491e-a95b-89442dd97e43 None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/e0d08965bc9c4ca29812d304bfb4ab4f/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.785791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f7bbf-c36c-491e-a95b-89442dd97e43 None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=e0d08965bc9c4ca29812d304bfb4ab4f, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:11.788301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f7bbf-c36c-491e-a95b-89442dd97e43 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:11.813689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 406/806] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:36:11 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/e0d08965bc9c4ca29812d304bfb4ab4f/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:36:11.817085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f7bbf-c36c-491e-a95b-89442dd97e43 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.825076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a23860-7f7b-4ec1-bcda-9e6b7bb132b9 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:11.869432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:11.932195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5e336-4d6f-4b61-a323-4157507a0678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:11.932816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e336-4d6f-4b61-a323-4157507a0678 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.933122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e336-4d6f-4b61-a323-4157507a0678 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.933409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5e336-4d6f-4b61-a323-4157507a0678 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.934055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5e336-4d6f-4b61-a323-4157507a0678 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:11.934365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5e336-4d6f-4b61-a323-4157507a0678 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:11.935876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5e336-4d6f-4b61-a323-4157507a0678 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:11.943157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 401/807] 10.222.0.15 () {56 vars in 1124 bytes} [Fri Sep 9 01:36:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:11.949095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d31a-e27e-4d28-a4b4-5d79b8f8bd49 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:11.949432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d31a-e27e-4d28-a4b4-5d79b8f8bd49 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:11.949715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320d31a-e27e-4d28-a4b4-5d79b8f8bd49 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:11.951556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 402/808] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:11.952370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320d31a-e27e-4d28-a4b4-5d79b8f8bd49 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:11.966827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f7bbf-c36c-491e-a95b-89442dd97e43 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:12.007877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0510a-a7f4-43fc-9581-45b274561b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:12.008487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0510a-a7f4-43fc-9581-45b274561b3f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.008760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0510a-a7f4-43fc-9581-45b274561b3f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.009028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0510a-a7f4-43fc-9581-45b274561b3f None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.009785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0510a-a7f4-43fc-9581-45b274561b3f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:12.010088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0510a-a7f4-43fc-9581-45b274561b3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:12.011581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0510a-a7f4-43fc-9581-45b274561b3f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:12.032390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 407/809] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:36:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:12.034401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7056de-e2d2-4ee7-ad69-afedd4e4189a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.034682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7056de-e2d2-4ee7-ad69-afedd4e4189a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.034952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7056de-e2d2-4ee7-ad69-afedd4e4189a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.043977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:12.057918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-0d7056de-e2d2-4ee7-ad69-afedd4e4189a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:12.059195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0d7056de-e2d2-4ee7-ad69-afedd4e4189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:12.085709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320d31a-e27e-4d28-a4b4-5d79b8f8bd49 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:12.122375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a0bd3-b382-47f5-b843-d37d1d2519dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:12.123127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a0bd3-b382-47f5-b843-d37d1d2519dd None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.123406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a0bd3-b382-47f5-b843-d37d1d2519dd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.123786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a0bd3-b382-47f5-b843-d37d1d2519dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.124442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a0bd3-b382-47f5-b843-d37d1d2519dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:12.125817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a0bd3-b382-47f5-b843-d37d1d2519dd None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:12.128329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 408/810] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:12.128980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7056de-e2d2-4ee7-ad69-afedd4e4189a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:12.169089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 403/811] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:36:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:12.169332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:12.169425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a0bd3-b382-47f5-b843-d37d1d2519dd None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:12.309771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7056de-e2d2-4ee7-ad69-afedd4e4189a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:12.375624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9adab-2c29-4194-92a6-b8087dbb9439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:12.376259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9adab-2c29-4194-92a6-b8087dbb9439 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.376533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9adab-2c29-4194-92a6-b8087dbb9439 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.376808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9adab-2c29-4194-92a6-b8087dbb9439 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.386130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a0bd3-b382-47f5-b843-d37d1d2519dd None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:12.394910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9adab-2c29-4194-92a6-b8087dbb9439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-fac9adab-2c29-4194-92a6-b8087dbb9439 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:12.401824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:12.402800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:36:12.403629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:12.404256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 409/812] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:36:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:12.412843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:12.415920 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9adab-2c29-4194-92a6-b8087dbb9439 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:12.430292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c928e-d534-447b-bcc8-98ce9225fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:12.430904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c928e-d534-447b-bcc8-98ce9225fbed None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.431179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c928e-d534-447b-bcc8-98ce9225fbed None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.431447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c928e-d534-447b-bcc8-98ce9225fbed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.498762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:12.514926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c928e-d534-447b-bcc8-98ce9225fbed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:12.516460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c928e-d534-447b-bcc8-98ce9225fbed None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:12.538297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9adab-2c29-4194-92a6-b8087dbb9439 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:12.575485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:12.576092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.576367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.576629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.577324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:12.577622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:12.578971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:12.588176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 410/813] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:36:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:12.589115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:12.645086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 404/814] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:12.651095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c928e-d534-447b-bcc8-98ce9225fbed None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:12.679328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:12.698294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c478785-2207-4cda-8840-db68b4d48154 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:12.724946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:12.735379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:12.736148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.736530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.736900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] PATH_INFO: `/v3/services/network` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.737629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:12.739223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:12.750638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:12.751864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 01:36:12.752741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:12.753632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 411/815] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:12 2022] GET /identity/v3/services/network => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:12.754761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:12.772179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c928e-d534-447b-bcc8-98ce9225fbed None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:12.819544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:12.820611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.821119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.821702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.833237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:12.834918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:12.852175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:12.853184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:12.854071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:12.854071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:12.854071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:12.856208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 405/816] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:12.857371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:12.881395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c274dff-d7a5-4f88-a55a-3b59b0118aa6 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:12.922254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:12.922928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:12.923207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:12.923479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:12.936769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:12.938907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:12.955045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:12.956071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:12.956965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:12.956965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:12.956965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:12.961510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 412/817] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:12.962497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.017395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:13.077364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:13.080004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc0411-489a-402b-b6a8-28b18d89a231 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.107907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44edb94-950e-47b9-ae0f-f1dfd0968660 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.118905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.119556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.119855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.120391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.120740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:13.121039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.122592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.137141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 413/818] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:13.138278 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.155682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.156773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.157283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.157756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.158707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] RBAC: Adding query filter params (name=network) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:13.159321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.160880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.167314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 406/819] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:13.168368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.262037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81096a5d-7071-4a5e-942e-52f9ba976335 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.265602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:13.302554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.303168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.303296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a2179-eaef-42ef-8b5f-ee6dec216010 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.303463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.303755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.304471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:13.305334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.306776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.324795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 414/820] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:13.326517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.350636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.351255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.352056 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.352323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.362850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.364458 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.366907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:13.383627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:13.384625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:13.385590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:13.392921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 407/821] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:36:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:13.394013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.439712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f17f64-9123-49ba-b236-06e188679388 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.476970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.477604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.477874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.478136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.478767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] RBAC: Adding query filter params (type=network) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:13.479047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.480618 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.491081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 415/822] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:13.491712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.504693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6621b0c-2cf2-49f1-b24c-e071f5a8b31a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.522958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:13.544380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.545389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.546003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.546489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.554060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.556000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:13.568373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:13.569346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:13.570250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:13.570250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:13.570250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:13.570250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:13.570250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:13.570250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:13.570250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:13.570250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:13.577271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 408/823] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:13.578328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.598887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d1a8b-7d86-476e-a801-5268ea836f26 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.636271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.636786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.637073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.637351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.638064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:13.638365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.639762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.645390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:13.663616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 416/824] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 01:36:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:13.663616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.700991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df8b71-2a7d-4dc4-a599-5b2b895f31fb None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.744367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16185a3-6d54-4f0e-a3d5-c84ab6eee908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.744969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16185a3-6d54-4f0e-a3d5-c84ab6eee908 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.745285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16185a3-6d54-4f0e-a3d5-c84ab6eee908 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.745552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16185a3-6d54-4f0e-a3d5-c84ab6eee908 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.746264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16185a3-6d54-4f0e-a3d5-c84ab6eee908 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.748020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16185a3-6d54-4f0e-a3d5-c84ab6eee908 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.757774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:13.781138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 409/825] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:36:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:13.781138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16185a3-6d54-4f0e-a3d5-c84ab6eee908 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.783993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dc600-504c-46ea-ae26-38d975b0343d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.858603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.859704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.860213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.860708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:13.861628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:13.862134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:13.863750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:13.889042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 417/826] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:13.895258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:13.906400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16185a3-6d54-4f0e-a3d5-c84ab6eee908 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:13.939883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:13.993045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:13.993723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:13.993996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:13.994261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.023919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:14.026145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:14.044084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:14.045104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:14.045992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:14.045992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:14.045992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:14.045992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 410/827] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:14.051409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:14.092285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1b2f-2ddf-49a1-bd77-67c4de3a58a7 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:14.132245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e4fe4-a6c1-4988-8410-2df9805da638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:14.133038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e4fe4-a6c1-4988-8410-2df9805da638 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.133406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e4fe4-a6c1-4988-8410-2df9805da638 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.133739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e4fe4-a6c1-4988-8410-2df9805da638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.134521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e4fe4-a6c1-4988-8410-2df9805da638 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:14.134888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e4fe4-a6c1-4988-8410-2df9805da638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:14.136357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e4fe4-a6c1-4988-8410-2df9805da638 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:14.145430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 418/828] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:13 2022] GET /identity/v3/role_assignments?user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=b131b61299154544a627315649090a49 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:14.146620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ed008-dc60-4cd4-b736-79febb148f2b None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.146903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ed008-dc60-4cd4-b736-79febb148f2b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.147173 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ed008-dc60-4cd4-b736-79febb148f2b None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.148549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 419/829] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:14.156009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022c872-d083-4c2e-b63f-57a66049a7ad None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.156009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022c872-d083-4c2e-b63f-57a66049a7ad None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.156009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022c872-d083-4c2e-b63f-57a66049a7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.171178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:14.178720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b022c872-d083-4c2e-b63f-57a66049a7ad None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:14.183526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b022c872-d083-4c2e-b63f-57a66049a7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:14.189516 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16788894-b782-41d9-aae6-cc9e0b8f3d50 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:14.236614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:14.238343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.238777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.239193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.240011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:14.241008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:14.243728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:14.262692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 411/830] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:14.268504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:14.277686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 420/831] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:14.301445 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:14.382262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b7989-a09f-4680-8298-137e278c3f7f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:14.419098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21a6cb-769a-449c-a97d-b898d6f5f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:14.419652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21a6cb-769a-449c-a97d-b898d6f5f40d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.419974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21a6cb-769a-449c-a97d-b898d6f5f40d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.420257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21a6cb-769a-449c-a97d-b898d6f5f40d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.427286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21a6cb-769a-449c-a97d-b898d6f5f40d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:14.428698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21a6cb-769a-449c-a97d-b898d6f5f40d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-ca21a6cb-769a-449c-a97d-b898d6f5f40d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:14.440341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:14.441322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:14.442307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 412/832] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:14.449095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022c872-d083-4c2e-b63f-57a66049a7ad None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:14.553328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022c872-d083-4c2e-b63f-57a66049a7ad None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:14.567053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a557ec3-3672-4218-b66a-b3b68c986275 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.567380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a557ec3-3672-4218-b66a-b3b68c986275 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.567574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a557ec3-3672-4218-b66a-b3b68c986275 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.569038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 413/833] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:14.570856 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a557ec3-3672-4218-b66a-b3b68c986275 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:14.592180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291843fc-62c8-4ebc-9f58-7b7b89e02fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:14.593103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291843fc-62c8-4ebc-9f58-7b7b89e02fbd None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.593394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291843fc-62c8-4ebc-9f58-7b7b89e02fbd None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.593667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291843fc-62c8-4ebc-9f58-7b7b89e02fbd None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.594352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291843fc-62c8-4ebc-9f58-7b7b89e02fbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:14.594657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291843fc-62c8-4ebc-9f58-7b7b89e02fbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:14.596502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291843fc-62c8-4ebc-9f58-7b7b89e02fbd None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:14.599644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:14.616154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 421/834] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:14.616154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ff681-79fa-4a1f-9018-c22a305560a8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.616154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ff681-79fa-4a1f-9018-c22a305560a8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.616154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ff681-79fa-4a1f-9018-c22a305560a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.636071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-203ff681-79fa-4a1f-9018-c22a305560a8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:14.637953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-203ff681-79fa-4a1f-9018-c22a305560a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:14.665560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:14.683287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a557ec3-3672-4218-b66a-b3b68c986275 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:14.714317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 422/835] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:14.719853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03458cd4-c112-46ef-87a7-4f33afc2e42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:14.720548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458cd4-c112-46ef-87a7-4f33afc2e42f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.720897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458cd4-c112-46ef-87a7-4f33afc2e42f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.721268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03458cd4-c112-46ef-87a7-4f33afc2e42f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.725948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ff681-79fa-4a1f-9018-c22a305560a8 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:14.732531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03458cd4-c112-46ef-87a7-4f33afc2e42f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-03458cd4-c112-46ef-87a7-4f33afc2e42f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:14.737521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:14.738645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:14.739699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:14.746365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 414/836] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:14.747449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03458cd4-c112-46ef-87a7-4f33afc2e42f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:14.848982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ff681-79fa-4a1f-9018-c22a305560a8 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:14.869792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03458cd4-c112-46ef-87a7-4f33afc2e42f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:14.876968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:14.887862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641af4f-0290-4c77-b84b-8697506fc41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:14.888659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641af4f-0290-4c77-b84b-8697506fc41a None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.889035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641af4f-0290-4c77-b84b-8697506fc41a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.889524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641af4f-0290-4c77-b84b-8697506fc41a None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/9cfe3dd0af154707bdaae5b52cbd6539/roles/a48f239bb178461d9d0ab9daaef7e154` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.908267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641af4f-0290-4c77-b84b-8697506fc41a None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=9cfe3dd0af154707bdaae5b52cbd6539, role_id=a48f239bb178461d9d0ab9daaef7e154)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:14.909917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2641af4f-0290-4c77-b84b-8697506fc41a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:14.917436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:14.921300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:14.922182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:14.923510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:14.924260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:14.925388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:14.926783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:14.935817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 423/837] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:36:14 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/9cfe3dd0af154707bdaae5b52cbd6539/roles/a48f239bb178461d9d0ab9daaef7e154 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:36:14.951355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 415/838] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:14.957258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641af4f-0290-4c77-b84b-8697506fc41a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:14.991977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:15.008743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:15.101941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2641af4f-0290-4c77-b84b-8697506fc41a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:15.146455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05273e-b259-4a98-8ae2-59fd6fdc180b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:15.149418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:15.149418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:15.149837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:15.150195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:15.161431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:15.162993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:15.180105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:15.181375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:15.182416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:15.182416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:15.182416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:15.182416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 424/839] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:15.189537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:15.191524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:15.192644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:15.193115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:15.194489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:15.195857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:15.197476 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:15.210164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:15.211440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:15.212502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:15.220449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 416/840] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:15.220825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:15.230443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:15.312071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcd395-5796-421d-8871-51e94dcdeabd None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:15.325023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:15.347435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:15.348040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:15.348314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:15.348582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:15.349247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:15.349549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:15.350897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:15.359257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae18ad5-eaa7-4a94-b1c4-7f9269ec3318 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:15.367995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 425/841] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:15.369281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:15.398060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:15.399290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:15.399571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:15.399941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:15.400661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:15.400968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:15.402612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:15.411133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 417/842] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:15.417189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:15.471440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:15.486682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e24fd-79f6-4fd9-97ea-edf4bff3122e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:15.535197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:15.536205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:15.536776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:15.537273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:15.545712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:15.547513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:15.549153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb0920-b182-4536-aa3b-b35d572c1d78 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:15.560808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:15.561829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 01:36:15.562758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:15.565178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 426/843] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:15.571428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:15.594134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:15.594866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:15.595248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:15.595811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:15.596843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:15.597415 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:15.598880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:15.602022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:15.609366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 418/844] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:15.694683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedb98d-4209-4640-8794-e98d03cb89b6 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:15.732753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:15.733629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:15.734048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:15.734460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:15.735333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:15.735801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:15.737411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:15.762416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 427/845] 10.222.0.15 () {56 vars in 1135 bytes} [Fri Sep 9 01:36:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:15.768517 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:15.878645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18c276-448f-49cb-ae5e-42bccdfbb1f9 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:15.922197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9938e9-cfcc-464b-bd6e-87c728d25cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:15.923233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9938e9-cfcc-464b-bd6e-87c728d25cc4 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:15.923739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9938e9-cfcc-464b-bd6e-87c728d25cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:15.924265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9938e9-cfcc-464b-bd6e-87c728d25cc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:15.996498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9938e9-cfcc-464b-bd6e-87c728d25cc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:15.997075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:16.001761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9938e9-cfcc-464b-bd6e-87c728d25cc4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:16.105209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 419/846] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:15 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:16.107516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:16.116544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:16.215426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa19fb-4aa1-43f1-8457-0ecf02701d01 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:16.250944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:16.251937 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:16.252420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:16.252889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:16.264241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:16.265988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:16.281988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:16.282977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:16.283854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:16.283854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:16.283854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:16.286064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 428/847] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:16.292036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9938e9-cfcc-464b-bd6e-87c728d25cc4 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:16.395882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:16.398865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9938e9-cfcc-464b-bd6e-87c728d25cc4 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:16.434332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca906f7-8e23-4b62-b65d-3a2eabe13418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:16.434985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca906f7-8e23-4b62-b65d-3a2eabe13418 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:16.435273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca906f7-8e23-4b62-b65d-3a2eabe13418 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:16.435540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca906f7-8e23-4b62-b65d-3a2eabe13418 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:16.436188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca906f7-8e23-4b62-b65d-3a2eabe13418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:16.436493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca906f7-8e23-4b62-b65d-3a2eabe13418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:16.437927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca906f7-8e23-4b62-b65d-3a2eabe13418 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:16.452212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 420/848] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:16.458901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:16.521727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:16.571928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e210cd3-58c2-4591-87c2-0b2269e09f90 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:16.588976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c35c-c354-423a-aac3-57152f455480 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:16.589375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c35c-c354-423a-aac3-57152f455480 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:16.589751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342c35c-c354-423a-aac3-57152f455480 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:16.591425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 421/849] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:16.596694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d558e34-5bf6-4da2-afb5-01518cf51408 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:16.597693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d558e34-5bf6-4da2-afb5-01518cf51408 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:16.598174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d558e34-5bf6-4da2-afb5-01518cf51408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:16.612861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:16.613588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:16.613871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:16.614150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:16.622882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1d558e34-5bf6-4da2-afb5-01518cf51408 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:16.624164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:16.625645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:16.625929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1d558e34-5bf6-4da2-afb5-01518cf51408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:16.638676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:16.639577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:16.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 429/850] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:16.645742 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:16.717941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 422/851] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:16.758739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed5df4-f6bf-46cb-8e44-60204755e277 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:16.776531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:16.794118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:16.794727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:16.795013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:16.795332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:16.796025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:16.796336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:16.797829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:16.816037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 430/852] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:16.816919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d558e34-5bf6-4da2-afb5-01518cf51408 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:16.911429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:16.925907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d558e34-5bf6-4da2-afb5-01518cf51408 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:16.960651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:16.961554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:16.962023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:16.962439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:16.963307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=e0d08965bc9c4ca29812d304bfb4ab4f) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:16.963774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:16.965366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:16.971728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 423/853] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:16 2022] GET /identity/v3/role_assignments?user.id=e0d08965bc9c4ca29812d304bfb4ab4f&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=b131b61299154544a627315649090a49 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:17.001765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:17.115832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0499-d807-4b37-9f00-227623a994f6 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:17.156125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e6b8f-189c-4088-8b37-47bbb467e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:17.156792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6b8f-189c-4088-8b37-47bbb467e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:17.157092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6b8f-189c-4088-8b37-47bbb467e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:17.157375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6b8f-189c-4088-8b37-47bbb467e8d8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:17.158002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e6b8f-189c-4088-8b37-47bbb467e8d8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:17.159398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e6b8f-189c-4088-8b37-47bbb467e8d8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-250e6b8f-189c-4088-8b37-47bbb467e8d8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:17.169909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:17.171081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:17.172140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 431/854] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 9 01:36:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:17.179098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:17.262052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830ecb-5d8f-4bfd-9a3c-01d6c3e4bdee None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:17.262702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830ecb-5d8f-4bfd-9a3c-01d6c3e4bdee None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:17.263243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc830ecb-5d8f-4bfd-9a3c-01d6c3e4bdee None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:17.265131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 432/855] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:17.275051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce969-ac80-48d1-81b2-ff016c507098 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:17.275614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce969-ac80-48d1-81b2-ff016c507098 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:17.276096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce969-ac80-48d1-81b2-ff016c507098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:17.295608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-748ce969-ac80-48d1-81b2-ff016c507098 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:17.296805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-748ce969-ac80-48d1-81b2-ff016c507098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:17.300203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745a3cb-01e7-4156-82d3-d195fc581174 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:17.339759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:17.348899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:17.349916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:17.350799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:17.351610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:17.352449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:17.353220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:17.355636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:17.362141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 424/856] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:17.369300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:17.383670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 433/857] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:17.396496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:17.483816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bc7a7-fc6b-472b-bc1b-0c117f490f99 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:17.521887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:17.522773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:17.523740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:17.524155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:17.524921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:17.525386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:17.526908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:17.531983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 425/858] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:17.661679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ce969-ac80-48d1-81b2-ff016c507098 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:17.774360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ce969-ac80-48d1-81b2-ff016c507098 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:17.811154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8bc98-268a-4f49-a29d-7d6764a97c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:17.812064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8bc98-268a-4f49-a29d-7d6764a97c0f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:17.812487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8bc98-268a-4f49-a29d-7d6764a97c0f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:17.814167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8bc98-268a-4f49-a29d-7d6764a97c0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:17.824677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8bc98-268a-4f49-a29d-7d6764a97c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-bec8bc98-268a-4f49-a29d-7d6764a97c0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:17.829733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:17.830857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:36:17.831904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:17.834169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 434/859] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:36:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:17.841094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:17.869879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7a68-05aa-40f3-9e86-24917187ba23 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:17.870156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7a68-05aa-40f3-9e86-24917187ba23 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:17.870432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7a68-05aa-40f3-9e86-24917187ba23 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:17.872593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 435/860] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:17.877535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86126-3620-4508-bc95-ece195d10abc None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:17.877819 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86126-3620-4508-bc95-ece195d10abc None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:17.878085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86126-3620-4508-bc95-ece195d10abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:17.900789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e2b86126-3620-4508-bc95-ece195d10abc None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:17.902015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e2b86126-3620-4508-bc95-ece195d10abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:17.934367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:17.949180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:17.977487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096096bf-cfa7-4a98-ae5a-6dfb65206c64 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:17.993192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 436/861] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:18.025582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:18.026563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:18.027046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:18.027519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:18.028370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:18.028894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:18.030463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:18.039889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 426/862] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:36:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:18.045824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b86126-3620-4508-bc95-ece195d10abc None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:18.147437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b86126-3620-4508-bc95-ece195d10abc None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:18.183532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:18.184547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:18.185030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:18.185535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:18.196011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:18.197780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:18.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:18.214151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:18.215134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:18.215134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:18.215134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:18.221369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 437/863] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:18.223267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:18.261212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:18.284427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:18.354749 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e5de5-2f12-4342-8c32-2165a6b0ac97 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:18.375470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241379a8-47e3-4051-afd0-6cb2f0f11126 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:18.391824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:18.392760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:18.393216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:18.393647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:18.394493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:18.395017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:18.396671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:18.410534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 427/864] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:18.416464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:18.421275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:18.426724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f27a4-b1c7-4f72-9d09-097fe41e1102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:18.428436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f27a4-b1c7-4f72-9d09-097fe41e1102 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:18.428924 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f27a4-b1c7-4f72-9d09-097fe41e1102 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:18.430427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f27a4-b1c7-4f72-9d09-097fe41e1102 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:18.431744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f27a4-b1c7-4f72-9d09-097fe41e1102 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:18.433727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f27a4-b1c7-4f72-9d09-097fe41e1102 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:18.456502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 438/865] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:36:18 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:36:18.542547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e463b4-949e-4147-8717-bfd92e83b8c4 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:18.589313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:18.590092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:18.590447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:18.590718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:18.599320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:18.600849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:18.614252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:18.615258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:18.616182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 428/866] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:36:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:18.623833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f27a4-b1c7-4f72-9d09-097fe41e1102 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:18.705131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:18.733033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f27a4-b1c7-4f72-9d09-097fe41e1102 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:18.767238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:18.767837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:18.768116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:18.768385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:18.769133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:18.769443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:18.770857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:18.788541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 439/867] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 01:36:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:18.794733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:18.859577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:18.907432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836a149-b712-4523-ba97-39de8f569929 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:18.943435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f9c57-36bb-4761-9deb-a36701b738cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:18.944053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f9c57-36bb-4761-9deb-a36701b738cb None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:18.944331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f9c57-36bb-4761-9deb-a36701b738cb None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:18.944646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f9c57-36bb-4761-9deb-a36701b738cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.018476 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f9c57-36bb-4761-9deb-a36701b738cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:19.019843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f9c57-36bb-4761-9deb-a36701b738cb None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:19.079635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:19.122207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 429/868] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:18 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:19.127715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:19.236312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e261764-b8ef-48fb-aae0-8a18f87f7d7c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:19.270430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:19.275775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b30903-4f64-489c-a22f-b9dcd3f83fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:19.276412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b30903-4f64-489c-a22f-b9dcd3f83fea None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.276684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b30903-4f64-489c-a22f-b9dcd3f83fea None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.276955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b30903-4f64-489c-a22f-b9dcd3f83fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.287307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b30903-4f64-489c-a22f-b9dcd3f83fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:19.288848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b30903-4f64-489c-a22f-b9dcd3f83fea None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-79b30903-4f64-489c-a22f-b9dcd3f83fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:19.304242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:19.305415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:19.306379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:19.306379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:19.306379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:19.306379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 440/869] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:19.312940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f9c57-36bb-4761-9deb-a36701b738cb None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:19.341494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78229-f67e-4246-add3-d53b1ffb8d61 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.341832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78229-f67e-4246-add3-d53b1ffb8d61 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.342035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78229-f67e-4246-add3-d53b1ffb8d61 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.343648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 441/870] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:19.348574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8cc02-cf80-40dc-b60f-11ebbd428c11 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.348848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8cc02-cf80-40dc-b60f-11ebbd428c11 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.349139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8cc02-cf80-40dc-b60f-11ebbd428c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.367437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ccf8cc02-cf80-40dc-b60f-11ebbd428c11 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:19.368622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ccf8cc02-cf80-40dc-b60f-11ebbd428c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:19.420199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f9c57-36bb-4761-9deb-a36701b738cb None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:19.447088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 442/871] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:19.463664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:19.464647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.465188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.465698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.466572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:19.467104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:19.468777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:19.486790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 430/872] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:19.492451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8cc02-cf80-40dc-b60f-11ebbd428c11 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:19.599147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8cc02-cf80-40dc-b60f-11ebbd428c11 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:19.614716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:19.637196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:19.637831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.638105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.638375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.645142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:19.646632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:19.657840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:19.658738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:19.659632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 443/873] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:19.666420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:19.696306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:19.766755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:19.790410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b662bbf-9c19-485c-bd39-8f8068bda5c0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:19.831794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b8657-b0ff-4048-835b-bd4d82e37a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:19.832568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b8657-b0ff-4048-835b-bd4d82e37a71 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.833194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b8657-b0ff-4048-835b-bd4d82e37a71 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.833487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b8657-b0ff-4048-835b-bd4d82e37a71 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.834467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b8657-b0ff-4048-835b-bd4d82e37a71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:19.834780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b8657-b0ff-4048-835b-bd4d82e37a71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:19.837703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b8657-b0ff-4048-835b-bd4d82e37a71 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:19.852962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 431/874] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:19.858923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e812-ba78-4998-96f5-ab0824e3331e None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.859209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e812-ba78-4998-96f5-ab0824e3331e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.859487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757e812-ba78-4998-96f5-ab0824e3331e None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.860864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 432/875] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:19.861455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e812-ba78-4998-96f5-ab0824e3331e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:19.883100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebd417-ae9b-47ba-a40a-c01e4c7f0cc5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:19.921667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3d850-12ce-452e-bdc0-1a2f268b8262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:19.922541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3d850-12ce-452e-bdc0-1a2f268b8262 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.922674 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3d850-12ce-452e-bdc0-1a2f268b8262 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.922944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3d850-12ce-452e-bdc0-1a2f268b8262 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.923534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3d850-12ce-452e-bdc0-1a2f268b8262 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:19.924900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3d850-12ce-452e-bdc0-1a2f268b8262 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-1ee3d850-12ce-452e-bdc0-1a2f268b8262 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:19.934096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:19.935355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:19.936425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 444/876] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:19.937457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80e4aa-c47f-4cb0-8f39-159fddf11906 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:19.937457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80e4aa-c47f-4cb0-8f39-159fddf11906 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:19.937457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80e4aa-c47f-4cb0-8f39-159fddf11906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:19.955799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6f80e4aa-c47f-4cb0-8f39-159fddf11906 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:19.956982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6f80e4aa-c47f-4cb0-8f39-159fddf11906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:19.961357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:19.984230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757e812-ba78-4998-96f5-ab0824e3331e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:19.988953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:20.022668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:20.023813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:20.024092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:20.024362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:20.025099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=9cfe3dd0af154707bdaae5b52cbd6539) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:20.025424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:20.026830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:20.029172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 445/877] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:19 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:20.030168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80e4aa-c47f-4cb0-8f39-159fddf11906 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:20.042116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 433/878] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:19 2022] GET /identity/v3/role_assignments?user.id=9cfe3dd0af154707bdaae5b52cbd6539&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:20.143823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80e4aa-c47f-4cb0-8f39-159fddf11906 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:20.181306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:20.182006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:20.182285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:20.182559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:20.183257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:20.183559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:20.184963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:20.190699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 446/879] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:20.196882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:20.311699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174020d6-d370-4eae-9e37-abf8aabef505 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:20.312919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:20.350042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8122c87-69e7-49d1-a22e-d1c433be3562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:20.350679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8122c87-69e7-49d1-a22e-d1c433be3562 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:20.350966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8122c87-69e7-49d1-a22e-d1c433be3562 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:20.351852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8122c87-69e7-49d1-a22e-d1c433be3562 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:20.352481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8122c87-69e7-49d1-a22e-d1c433be3562 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:20.352776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8122c87-69e7-49d1-a22e-d1c433be3562 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:20.354176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8122c87-69e7-49d1-a22e-d1c433be3562 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:20.361169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 434/880] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:20.361776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:20.447706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19962b83-1371-4126-955d-2b6ee86e10f9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:20.486521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439f95a-9617-4550-bad6-a781e283ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:20.487528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439f95a-9617-4550-bad6-a781e283ca34 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:20.488034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439f95a-9617-4550-bad6-a781e283ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:20.488510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439f95a-9617-4550-bad6-a781e283ca34 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:20.489453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439f95a-9617-4550-bad6-a781e283ca34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:20.491078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439f95a-9617-4550-bad6-a781e283ca34 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:20.511932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 447/881] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:36:20 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:36:20.811508 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc77d92-86d1-43d9-a441-a05f26c98fd1 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:20.811709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc77d92-86d1-43d9-a441-a05f26c98fd1 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:20.811976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc77d92-86d1-43d9-a441-a05f26c98fd1 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:20.813587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 435/882] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:20.820631 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed294-e4eb-4ea9-b67c-edb4f740e50c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:20.820916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed294-e4eb-4ea9-b67c-edb4f740e50c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:20.821233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ed294-e4eb-4ea9-b67c-edb4f740e50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:20.840497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b02ed294-e4eb-4ea9-b67c-edb4f740e50c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:20.842130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b02ed294-e4eb-4ea9-b67c-edb4f740e50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:20.919854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 448/883] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:21.186000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc77d92-86d1-43d9-a441-a05f26c98fd1 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:21.198195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:21.302713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc77d92-86d1-43d9-a441-a05f26c98fd1 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:21.338595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:21.339238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:21.339518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:21.339789 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:21.340367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:21.341764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:21.350739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:21.352334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:21.353213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 436/884] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:21.358457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ed294-e4eb-4ea9-b67c-edb4f740e50c None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:21.393606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:21.461965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ed294-e4eb-4ea9-b67c-edb4f740e50c None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:21.501356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:21.501972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:21.502251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:21.502523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:21.503153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:21.503457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:21.504814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:21.510739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 449/885] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:21.518138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:21.590629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:21.622603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eb4a4-02b0-40fa-858b-50bc8c44000c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:21.659104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:21.659708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:21.659981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:21.660332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:21.660963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:21.661334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:21.662754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:21.667909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 437/886] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:21.673970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:21.758111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:21.778064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92c8bb-1619-44c2-bd49-6a2280f4332b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:21.813858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a166f-1b1a-4a61-b3dd-66c7c676d0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:21.814489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a166f-1b1a-4a61-b3dd-66c7c676d0e9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:21.814762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a166f-1b1a-4a61-b3dd-66c7c676d0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:21.815030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a166f-1b1a-4a61-b3dd-66c7c676d0e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:21.815718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a166f-1b1a-4a61-b3dd-66c7c676d0e9 None admin] RBAC: Adding query filter params (interface=public, service_id=72c31f2d9fea4c93a4c1590fc945b847, region_id=RegionOne) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:21.816024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a166f-1b1a-4a61-b3dd-66c7c676d0e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:21.817387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a166f-1b1a-4a61-b3dd-66c7c676d0e9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:21.825379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 450/887] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 01:36:21 2022] GET /identity/v3/endpoints?service_id=72c31f2d9fea4c93a4c1590fc945b847&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:21.830777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:21.944599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd5709-a536-4147-838f-72c92c6ae2dc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:21.985346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776637b-e09a-48df-9966-a02e97a09a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:21.986309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776637b-e09a-48df-9966-a02e97a09a66 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:21.986816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776637b-e09a-48df-9966-a02e97a09a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:21.987291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776637b-e09a-48df-9966-a02e97a09a66 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:21.988192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776637b-e09a-48df-9966-a02e97a09a66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:21.989797 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776637b-e09a-48df-9966-a02e97a09a66 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:21.991889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:21.996917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 438/888] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:36:21 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:36:22.496708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a06aa-e994-42bd-a237-17eb6b2cc3fa None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:22.497784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a06aa-e994-42bd-a237-17eb6b2cc3fa None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:22.498277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a06aa-e994-42bd-a237-17eb6b2cc3fa None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:22.500246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 451/889] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:22.506607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11025a-0cdd-4566-a3f2-e91f3e43de91 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:22.508131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11025a-0cdd-4566-a3f2-e91f3e43de91 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:22.508610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11025a-0cdd-4566-a3f2-e91f3e43de91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:22.528350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-fd11025a-0cdd-4566-a3f2-e91f3e43de91 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:22.529967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fd11025a-0cdd-4566-a3f2-e91f3e43de91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:22.601978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 439/890] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:22.612486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123c041-740f-4df5-89c7-8cfb5c40e91e None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:22.612772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123c041-740f-4df5-89c7-8cfb5c40e91e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:22.613049 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123c041-740f-4df5-89c7-8cfb5c40e91e None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:22.614650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 452/891] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:22.619842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae644a-f1ec-48dd-9fec-975efc3a7d77 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:22.620365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae644a-f1ec-48dd-9fec-975efc3a7d77 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:22.620865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae644a-f1ec-48dd-9fec-975efc3a7d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:22.639746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-33ae644a-f1ec-48dd-9fec-975efc3a7d77 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:22.641288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-33ae644a-f1ec-48dd-9fec-975efc3a7d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:22.717490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 440/892] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:22.797987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f7e77-5166-4987-a9ab-0c7b9c9401bd None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:22.798179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f7e77-5166-4987-a9ab-0c7b9c9401bd None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:22.798437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f7e77-5166-4987-a9ab-0c7b9c9401bd None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:22.800064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 453/893] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:22.805864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec26d7-14da-4fc7-a542-9fcf844af9c0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:22.806149 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec26d7-14da-4fc7-a542-9fcf844af9c0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:22.806420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec26d7-14da-4fc7-a542-9fcf844af9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:22.825303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-81ec26d7-14da-4fc7-a542-9fcf844af9c0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:22.826470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-81ec26d7-14da-4fc7-a542-9fcf844af9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:22.884392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:22.897271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 441/894] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:22.899502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f7e77-5166-4987-a9ab-0c7b9c9401bd None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:22.994658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec26d7-14da-4fc7-a542-9fcf844af9c0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.018128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f7e77-5166-4987-a9ab-0c7b9c9401bd None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.054185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510e52-82fc-45db-bf72-d7d2c8e65e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.055181 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510e52-82fc-45db-bf72-d7d2c8e65e17 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.055664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510e52-82fc-45db-bf72-d7d2c8e65e17 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.056133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e510e52-82fc-45db-bf72-d7d2c8e65e17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.066907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e510e52-82fc-45db-bf72-d7d2c8e65e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-0e510e52-82fc-45db-bf72-d7d2c8e65e17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:23.072668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:23.073642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:23.074485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:23.081843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 454/895] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 01:36:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:23.082895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510e52-82fc-45db-bf72-d7d2c8e65e17 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.093340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:23.126512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec26d7-14da-4fc7-a542-9fcf844af9c0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.173650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc618df1-ca57-437c-b051-ebd87568cf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.174364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618df1-ca57-437c-b051-ebd87568cf0e None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.174651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618df1-ca57-437c-b051-ebd87568cf0e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.174923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618df1-ca57-437c-b051-ebd87568cf0e None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.175575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc618df1-ca57-437c-b051-ebd87568cf0e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.177435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc618df1-ca57-437c-b051-ebd87568cf0e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.197313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e510e52-82fc-45db-bf72-d7d2c8e65e17 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.201265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 442/896] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:36:22 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:36:23.201265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc618df1-ca57-437c-b051-ebd87568cf0e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.251698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.252317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.252595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.252863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.256449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:23.256449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.257941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.271591 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 455/897] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:23.277078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.292892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:23.359003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:23.368733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc618df1-ca57-437c-b051-ebd87568cf0e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.405662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.406386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.406672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.406944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.407554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.408876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.409406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ab1ec-d189-4c9a-86d8-09d3ad5ccd17 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:23.419254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:23.420212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:23.421107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 443/898] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:23.426084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.453799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.454573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.455354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.455354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.473696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.475628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:23.492871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:23.493872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:23.494770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:23.494770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:23.494770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:23.494770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 456/899] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:23.500391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.550130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b03d72-b640-4f3e-8806-a0132c00f804 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.568688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:23.603550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.604316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.604661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.605008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:23.606295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.607733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.618826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 444/900] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:23.619831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eded09-bddb-4785-941b-2df1de0268b1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.619966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.637508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:23.656157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.656777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.657336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.657476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.658066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:23.658382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.659713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.677703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 457/901] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:23.679037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.743610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26caed20-0d49-414f-8f07-f41e2e6f31ac None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.786812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.787572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.787702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aca434-4347-4700-afc9-c914b3eb2c7e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.788029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.788403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.789885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:23.790292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.792580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.798948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 445/902] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:23.804877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.829571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.830523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.830987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.831433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.839559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.841324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:23.854529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:23.855533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:23.856449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:23.859177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 458/903] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:23.866450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:23.868514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:23.911604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:23.938992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9616e-a6e8-421c-8cfe-069b95392bb1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.977180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:23.978065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:23.978412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:23.978754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:23.979725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] RBAC: Adding query filter params (interface=public, service_id=1bda0b8b714440b0b605d2ad8086c99c, region_id=RegionOne) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:23.980105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:23.980724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c85ed-94d3-4abe-b982-1c69be08067f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:23.981629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:23.992660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 446/904] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/endpoints?service_id=1bda0b8b714440b0b605d2ad8086c99c&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:23.993693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:24.017339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:24.018242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.018728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.019168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.020063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:24.020536 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:24.022188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:24.041268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 459/905] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:24.046520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:24.124387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bcfd1-a283-4e9d-b904-c3454f4c40e2 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:24.135409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:24.170025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3063b16-c30b-41f5-a248-13cfe245a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:24.171246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:24.171504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063b16-c30b-41f5-a248-13cfe245a97a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.172646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063b16-c30b-41f5-a248-13cfe245a97a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.173219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063b16-c30b-41f5-a248-13cfe245a97a None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.174072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3063b16-c30b-41f5-a248-13cfe245a97a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:24.175743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3063b16-c30b-41f5-a248-13cfe245a97a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:24.178055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da6d027-3f9b-4d8d-b6db-cf5eb6280e60 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:24.189530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 447/906] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:36:23 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:36:24.217759 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12c91e-d984-4df2-a08a-489459990ec1 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.218094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12c91e-d984-4df2-a08a-489459990ec1 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.218318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12c91e-d984-4df2-a08a-489459990ec1 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.219279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cfef2-7880-42f4-9a23-65e97cc48c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:24.220082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cfef2-7880-42f4-9a23-65e97cc48c68 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.220512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cfef2-7880-42f4-9a23-65e97cc48c68 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.220940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cfef2-7880-42f4-9a23-65e97cc48c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.224197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 448/907] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:24.226241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616b70-d876-4bfa-bc1c-790605beb2d6 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.226373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616b70-d876-4bfa-bc1c-790605beb2d6 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.226558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7616b70-d876-4bfa-bc1c-790605beb2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.247049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d7616b70-d876-4bfa-bc1c-790605beb2d6 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:24.248366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d7616b70-d876-4bfa-bc1c-790605beb2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:24.305143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cfef2-7880-42f4-9a23-65e97cc48c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:24.307583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cfef2-7880-42f4-9a23-65e97cc48c68 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:24.333635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 449/908] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:24.414822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:24.424015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 460/909] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:24 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 01:36:24.429963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7616b70-d876-4bfa-bc1c-790605beb2d6 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:24.531725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:24.535353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7616b70-d876-4bfa-bc1c-790605beb2d6 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:24.576219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:24.576965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.577161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.577438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.588951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:24.590428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:24.603534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cfef2-7880-42f4-9a23-65e97cc48c68 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:24.605751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:24.606683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:24.607560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:24.607560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:24.607560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:24.608468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 450/910] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:24.614989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:24.714826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cfef2-7880-42f4-9a23-65e97cc48c68 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:24.751007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:24.751608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fcd15-e452-4faa-b8c6-d0fcb3f9ef0e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:24.751724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.751901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.752167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.752810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:24.754209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:24.763733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:24.764703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 01:36:24.765635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:24.769173 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 461/911] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:24.771690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:24.792441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:24.793283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.793632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.793972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.794683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:24.795058 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:24.796591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:24.814958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 451/912] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:24.816315 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:24.831791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:24.838588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:24.889923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b901a55-7907-4b03-9c83-b40f8c577e79 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:24.930369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:24.931008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.931284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.931552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.932184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:24.932480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:24.934085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:24.939584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 462/913] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:24.941142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac57b2-29fb-4ec9-9962-61591f850d12 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:24.950603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:24.984686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:24.985837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:24.986706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:24.987418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:24.997169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:24.998870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:25.019139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:25.020274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:25.022204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:25.022204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:25.022204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:25.022204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:25.022204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:25.022204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:25.022204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:25.022204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:25.031447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 452/914] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:25.033888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:25.090775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb236ce-c7aa-46a6-8d6a-05bb8cfea18a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:25.112898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:25.121229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:25.137970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:25.139014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:25.139549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:25.140025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:25.140898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:25.141704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:25.144059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:25.148410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9913a-792f-45a8-b500-6ffaec278df7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:25.150511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 463/915] 10.222.0.15 () {56 vars in 1114 bytes} [Fri Sep 9 01:36:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:25.158013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:25.194724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:25.195769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:25.196258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:25.196738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:25.198826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:25.199432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:25.201153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:25.222829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 453/916] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:25.223851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:25.280213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79635d3-cd49-45ee-885c-8a4484617ff2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:25.318245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b5032-4ae0-47d6-a4f1-245b696a458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:25.318821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5032-4ae0-47d6-a4f1-245b696a458b None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:25.319100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5032-4ae0-47d6-a4f1-245b696a458b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:25.319370 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5032-4ae0-47d6-a4f1-245b696a458b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:25.320552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b5032-4ae0-47d6-a4f1-245b696a458b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:25.322046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b5032-4ae0-47d6-a4f1-245b696a458b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:25.350476 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dfece4-4e03-4373-9a37-daf109241929 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:25.353138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 464/917] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:36:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:25.390215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:25.402765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e84fcb-5337-4ea5-a70f-3fe75b6c5a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:25.403602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e84fcb-5337-4ea5-a70f-3fe75b6c5a02 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:25.403977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e84fcb-5337-4ea5-a70f-3fe75b6c5a02 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:25.404349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e84fcb-5337-4ea5-a70f-3fe75b6c5a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:25.405255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e84fcb-5337-4ea5-a70f-3fe75b6c5a02 None admin] RBAC: Adding query filter params (role.id=14e68dc1536843148f60370e1ea8013b, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=9cfe3dd0af154707bdaae5b52cbd6539) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:25.405704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e84fcb-5337-4ea5-a70f-3fe75b6c5a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:25.407360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e84fcb-5337-4ea5-a70f-3fe75b6c5a02 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:25.414980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 454/918] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:25 2022] GET /identity/v3/role_assignments?user.id=9cfe3dd0af154707bdaae5b52cbd6539&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=14e68dc1536843148f60370e1ea8013b => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:25.542027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59985a3a-b6f4-4189-a2c2-1969c5070c88 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:25.542390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59985a3a-b6f4-4189-a2c2-1969c5070c88 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:25.542763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59985a3a-b6f4-4189-a2c2-1969c5070c88 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:25.544697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 465/919] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:25.550427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe298f6-53ba-458d-b3b3-909757574745 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:25.550723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe298f6-53ba-458d-b3b3-909757574745 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:25.550990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe298f6-53ba-458d-b3b3-909757574745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:25.572360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9fe298f6-53ba-458d-b3b3-909757574745 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:25.573653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9fe298f6-53ba-458d-b3b3-909757574745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:25.646490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 455/920] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:25.917715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59985a3a-b6f4-4189-a2c2-1969c5070c88 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:25.966513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:26.027827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59985a3a-b6f4-4189-a2c2-1969c5070c88 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:26.061875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:26.062521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:26.062797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:26.063070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:26.063655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:26.065018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:26.074608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:26.075590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:26.076802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 466/921] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:26.081929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe298f6-53ba-458d-b3b3-909757574745 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:26.186709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe298f6-53ba-458d-b3b3-909757574745 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:26.211988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:26.224702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:26.225697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:26.226232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:26.226731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:26.227783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:26.228483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:26.230104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:26.235993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 456/922] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:26.241593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:26.348925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591a593-d534-4044-9e1d-702ccd5f1a9a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:26.357217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:26.391495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bbc385-fda1-4402-ad6c-34e58566484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:26.392115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbc385-fda1-4402-ad6c-34e58566484f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:26.392394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbc385-fda1-4402-ad6c-34e58566484f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:26.392666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbc385-fda1-4402-ad6c-34e58566484f None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:26.393333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bbc385-fda1-4402-ad6c-34e58566484f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:26.393640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bbc385-fda1-4402-ad6c-34e58566484f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:26.395032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bbc385-fda1-4402-ad6c-34e58566484f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:26.401778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 467/923] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:26.408028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:26.515096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447b8c-dcff-41cc-82d2-d0246db562fe None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:26.525424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ac220-eafb-49a5-8a8c-f013ecdc1ca8 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:26.526002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ac220-eafb-49a5-8a8c-f013ecdc1ca8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:26.526526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ac220-eafb-49a5-8a8c-f013ecdc1ca8 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:26.528494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 468/924] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:26.535757 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788d991-f171-4669-ad38-db9bcc7a4f7b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:26.536278 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788d991-f171-4669-ad38-db9bcc7a4f7b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:26.536641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788d991-f171-4669-ad38-db9bcc7a4f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:26.556826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:26.557700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:26.558115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:26.558560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:26.558917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-3788d991-f171-4669-ad38-db9bcc7a4f7b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:26.559571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] RBAC: Adding query filter params (interface=public, service_id=f0917153d0c344209f2d3a20234036bd, region_id=RegionOne) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:26.560032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:26.560384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3788d991-f171-4669-ad38-db9bcc7a4f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:26.561729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:26.567568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 457/925] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 01:36:26 2022] GET /identity/v3/endpoints?service_id=f0917153d0c344209f2d3a20234036bd&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:26.575204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:26.642337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:26.646195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 469/926] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:26.708593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b5c67-7f8b-40d1-bd44-0b51eaacf7d1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:26.744422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786a3d-4f3c-4853-871f-cdf4aab0c11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:26.745386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786a3d-4f3c-4853-871f-cdf4aab0c11d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:26.745850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786a3d-4f3c-4853-871f-cdf4aab0c11d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:26.746302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e786a3d-4f3c-4853-871f-cdf4aab0c11d None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:26.747092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e786a3d-4f3c-4853-871f-cdf4aab0c11d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:26.748629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e786a3d-4f3c-4853-871f-cdf4aab0c11d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:26.755397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 458/927] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:36:26 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:36:26.937188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788d991-f171-4669-ad38-db9bcc7a4f7b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:27.045710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:27.049745 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788d991-f171-4669-ad38-db9bcc7a4f7b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:27.085812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:27.086848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:27.087330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:27.087800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:27.088583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:27.090195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:27.099435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:27.100383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 01:36:27.101390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:27.104111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 470/928] 10.222.0.15 () {56 vars in 1132 bytes} [Fri Sep 9 01:36:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:27.110186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786a3d-4f3c-4853-871f-cdf4aab0c11d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:27.218943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e786a3d-4f3c-4853-871f-cdf4aab0c11d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:27.234761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:27.257977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:27.258932 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:27.259413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:27.259897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:27.260755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:27.261296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:27.262860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:27.268336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 459/929] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:27.274249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:27.385749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d892ab-ac26-490b-8716-cb5c4ffee5e9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:27.420795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8c95b-2b6c-4362-933c-3ed613b87529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:27.421434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8c95b-2b6c-4362-933c-3ed613b87529 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:27.421712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8c95b-2b6c-4362-933c-3ed613b87529 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:27.421980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8c95b-2b6c-4362-933c-3ed613b87529 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:27.422611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8c95b-2b6c-4362-933c-3ed613b87529 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:27.422911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8c95b-2b6c-4362-933c-3ed613b87529 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:27.424259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8c95b-2b6c-4362-933c-3ed613b87529 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:27.429988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 471/930] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:27.442428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:27.560901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb73d95-7feb-4082-97c9-e0db1bc2703a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:27.596358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab5fbb-bf81-43b0-acf1-4c1585ad348e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:27.596968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5fbb-bf81-43b0-acf1-4c1585ad348e None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:27.597281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5fbb-bf81-43b0-acf1-4c1585ad348e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:27.597609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab5fbb-bf81-43b0-acf1-4c1585ad348e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:27.598275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab5fbb-bf81-43b0-acf1-4c1585ad348e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:27.599625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab5fbb-bf81-43b0-acf1-4c1585ad348e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:27.623132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 460/931] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:36:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:27.649341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d5720-0cbe-4226-8d0f-5c71d85cd174 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:27.649666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d5720-0cbe-4226-8d0f-5c71d85cd174 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:27.649892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d5720-0cbe-4226-8d0f-5c71d85cd174 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:27.651665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 472/932] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:27.657596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52577-395b-4f7e-afbe-526baed03759 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:27.657896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52577-395b-4f7e-afbe-526baed03759 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:27.658193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b52577-395b-4f7e-afbe-526baed03759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:27.683655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-c0b52577-395b-4f7e-afbe-526baed03759 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:27.685018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c0b52577-395b-4f7e-afbe-526baed03759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:27.717554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:27.772496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 461/933] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:27.802270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e8fa-3b79-4f35-b9fa-c66ab61ed60d None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:27.802611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e8fa-3b79-4f35-b9fa-c66ab61ed60d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:27.802825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7e8fa-3b79-4f35-b9fa-c66ab61ed60d None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:27.804544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 473/934] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:27.809976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca13624-e9b7-457a-ad7b-e5440edf49ff None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:27.810265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca13624-e9b7-457a-ad7b-e5440edf49ff None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:27.810543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca13624-e9b7-457a-ad7b-e5440edf49ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:27.830038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4ca13624-e9b7-457a-ad7b-e5440edf49ff None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:27.831523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4ca13624-e9b7-457a-ad7b-e5440edf49ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:27.907733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 462/935] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:28.041976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7e8fa-3b79-4f35-b9fa-c66ab61ed60d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:28.072019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:28.151623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7e8fa-3b79-4f35-b9fa-c66ab61ed60d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:28.187210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ee815-6e2e-4d39-a6e2-b50046c951a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:28.187839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ee815-6e2e-4d39-a6e2-b50046c951a3 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:28.188110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ee815-6e2e-4d39-a6e2-b50046c951a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:28.188378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ee815-6e2e-4d39-a6e2-b50046c951a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:28.193947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca13624-e9b7-457a-ad7b-e5440edf49ff None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:28.201845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ee815-6e2e-4d39-a6e2-b50046c951a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-d80ee815-6e2e-4d39-a6e2-b50046c951a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:28.209208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:28.210214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:28.211575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:28.214541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 474/936] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 01:36:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:28.221860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ee815-6e2e-4d39-a6e2-b50046c951a3 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:28.302781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca13624-e9b7-457a-ad7b-e5440edf49ff None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:28.305482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:28.326723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:28.342612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:28.343087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ee815-6e2e-4d39-a6e2-b50046c951a3 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:28.343297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:28.343536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:28.343810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:28.344457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:28.346036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:28.348734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING keystone.server.flask.common [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 01:36:28.367656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.sql.core [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 01:36:28.367656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 01:36:28.367656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [parameters: {'id': 'f6b88f770c0a4b6fbc194e339f83e50a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 01:36:28.367656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87884) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 01:36:28.382862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:28.383783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:28.384267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:28.384756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:28.385648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:28.386178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6b88f770c0a4b6fbc194e339f83e50a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6b88f770c0a4b6fbc194e339f83e50a. Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application conn.query(q) Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 01:36:28.387400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result.read() Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 01:36:28.388159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application next(self.gen) Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application yield resource Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application next(self.gen) Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise self.value Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application session.commit() Sep 09 01:36:28.389104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.session.flush() Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application compat.raise_( Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 01:36:28.390001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 01:36:28.390899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application conn.query(q) Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result.read() Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 01:36:28.391697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [parameters: {'id': 'f6b88f770c0a4b6fbc194e339f83e50a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:28.392572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:28.393406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6b88f770c0a4b6fbc194e339f83e50a. Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:28.394320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 463/937] 10.222.0.15 () {60 vars in 1140 bytes} [Fri Sep 9 01:36:28 2022] POST /identity/v3/projects => generated 262 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:28.396273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:28.398623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:28.408689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 475/938] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 01:36:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:28.414126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:28.526559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d093ebb-00a5-4483-a014-dd68517f33b9 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:28.535381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cc942-26c0-4c34-91cf-68b294e241d1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:28.573868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:28.574790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:28.575072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:28.575760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:28.579579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:28.580517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:28.580812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:28.581095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:28.586761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:28.588807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:28.595397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:28.596956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:28.605779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:28.606685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:28.607556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:28.607556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:28.607556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:28.607556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:28.607556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:28.607556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:28.607556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:28.607556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:28.610485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 464/939] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:28.616323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:28.619486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:28.620403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:28.621295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:28.621295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:28.621295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:28.621295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 476/940] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:28.627152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:28.630268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:28.635597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:28.738570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb0ba1-ba54-4892-91e8-f5599eb9cb70 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:28.756040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf068d3-eedc-4d5e-afba-a5996a9d0f15 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:28.780501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:28.781299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:28.781675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:28.782043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:28.783249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:28.783648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:28.785635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:28.798721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:28.800125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:28.800614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:28.801046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:28.801908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:28.802360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:28.804986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:28.813205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 465/941] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:28.824115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 477/942] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:28.830258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:28.908539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:28.954341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fca58e-e565-48d7-bb4b-eb4bf6fd2d62 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:28.997717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346bfe1-b76d-4718-9481-79f7b635dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:28.998283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346bfe1-b76d-4718-9481-79f7b635dc9b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:28.998559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346bfe1-b76d-4718-9481-79f7b635dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:28.998832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346bfe1-b76d-4718-9481-79f7b635dc9b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.007900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346bfe1-b76d-4718-9481-79f7b635dc9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.009377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346bfe1-b76d-4718-9481-79f7b635dc9b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-a346bfe1-b76d-4718-9481-79f7b635dc9b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:29.022015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:29.023528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:29.024453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:29.028746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 466/943] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:36:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:29.029800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.064648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2644520-0b90-49fb-97b5-371dad11a5d2 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.064979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2644520-0b90-49fb-97b5-371dad11a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.065313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2644520-0b90-49fb-97b5-371dad11a5d2 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.066634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 467/944] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:29.072395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c71a-0002-4895-9d19-8edfb7f1d996 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.072672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c71a-0002-4895-9d19-8edfb7f1d996 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.072940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c71a-0002-4895-9d19-8edfb7f1d996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.097446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-42e6c71a-0002-4895-9d19-8edfb7f1d996 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:29.099280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-42e6c71a-0002-4895-9d19-8edfb7f1d996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:29.111076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:29.160766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1ac97-b2e8-40e2-9856-e49745057eb0 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:29.185176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 468/945] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:29.202249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:29.202865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.203201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.203473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.204201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:29.204503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.205949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:29.224397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 478/946] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 01:36:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:29.231240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c71a-0002-4895-9d19-8edfb7f1d996 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.292649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:29.339983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c71a-0002-4895-9d19-8edfb7f1d996 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:29.376243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:29.376877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.377185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.377461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.388977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.390469 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:29.406006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:29.406963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:29.407829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:29.407829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:29.407829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:29.407829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 469/947] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:29.413121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.452365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.529947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96cf6-9e23-4243-b146-f2e4f4b49c99 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:29.554951 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:29.575910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc9282-3714-48be-ba04-b3508e4948d7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:29.576691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:29.577331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.577606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.577875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.578509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:29.578810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.580164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:29.589219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:29.597460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 479/948] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:29.598696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.621394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:29.622828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.623301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.623749 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.624773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.626832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:29.639631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:29.640655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:29.641502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:29.647791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 470/949] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:29.651501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.725877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d789e9-d1b6-4836-8b3a-673ca4134622 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:29.766089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:29.767039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.767545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.768018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.777481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.779226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa7d91-2fb0-4cdc-87d9-6e4867819330 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:29.780002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:29.794773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:29.795760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:29.796664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:29.796664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:29.796664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:29.796664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:29.796664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:29.796664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:29.796664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:29.796664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:29.803936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 480/950] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:29.805769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.819881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:29.820586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.820871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.821169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.821798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:29.822100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.823453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:29.833216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 471/951] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:29.834690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.854719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:29.877009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:29.920492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98bf81-d138-49cc-ac08-2d42318ba70b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:29.953376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41947e99-d167-4af1-b284-e83ceeb70403 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:29.954855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2acef-a676-457a-8911-fca404ee64a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:29.956160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2acef-a676-457a-8911-fca404ee64a0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.956160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2acef-a676-457a-8911-fca404ee64a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.956160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2acef-a676-457a-8911-fca404ee64a0 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.956708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2acef-a676-457a-8911-fca404ee64a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:29.957016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2acef-a676-457a-8911-fca404ee64a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.958564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2acef-a676-457a-8911-fca404ee64a0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:29.970668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 481/952] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:29.971837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7a4cd-ccac-4927-9d4e-840ae9757a3d None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.972112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7a4cd-ccac-4927-9d4e-840ae9757a3d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.972379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7a4cd-ccac-4927-9d4e-840ae9757a3d None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.978423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 482/953] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:29.979518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7a4cd-ccac-4927-9d4e-840ae9757a3d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:29.991902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05e893-9d1c-488a-9053-854b28dad5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:29.992502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e893-9d1c-488a-9053-854b28dad5dd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:29.992778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e893-9d1c-488a-9053-854b28dad5dd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:29.993043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05e893-9d1c-488a-9053-854b28dad5dd None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:29.993690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05e893-9d1c-488a-9053-854b28dad5dd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:29.993988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05e893-9d1c-488a-9053-854b28dad5dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:29.995335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05e893-9d1c-488a-9053-854b28dad5dd None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:30.005046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 472/954] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:30.005046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412137-4179-418e-a003-8fb4d24adb3d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:30.005046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412137-4179-418e-a003-8fb4d24adb3d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:30.005046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412137-4179-418e-a003-8fb4d24adb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:30.022093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9c412137-4179-418e-a003-8fb4d24adb3d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:30.023395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9c412137-4179-418e-a003-8fb4d24adb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:30.081766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:30.090007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7a4cd-ccac-4927-9d4e-840ae9757a3d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:30.093950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 473/955] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:30.094914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c412137-4179-418e-a003-8fb4d24adb3d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:30.127429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:30.129871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b07160-d170-46c9-830b-0418dfe72484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:30.131601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07160-d170-46c9-830b-0418dfe72484 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:30.132021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07160-d170-46c9-830b-0418dfe72484 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:30.132464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07160-d170-46c9-830b-0418dfe72484 None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/9cfe3dd0af154707bdaae5b52cbd6539/roles/14e68dc1536843148f60370e1ea8013b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:30.135791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b07160-d170-46c9-830b-0418dfe72484 None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=9cfe3dd0af154707bdaae5b52cbd6539, role_id=14e68dc1536843148f60370e1ea8013b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:30.137563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b07160-d170-46c9-830b-0418dfe72484 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:30.161157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 483/956] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:36:29 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/9cfe3dd0af154707bdaae5b52cbd6539/roles/14e68dc1536843148f60370e1ea8013b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:36:30.233241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c412137-4179-418e-a003-8fb4d24adb3d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:30.271986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ddcb3-de9e-436d-af15-9f7eb3963678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:30.273197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ddcb3-de9e-436d-af15-9f7eb3963678 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:30.273457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ddcb3-de9e-436d-af15-9f7eb3963678 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:30.273725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ddcb3-de9e-436d-af15-9f7eb3963678 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:30.274394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ddcb3-de9e-436d-af15-9f7eb3963678 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:30.275766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ddcb3-de9e-436d-af15-9f7eb3963678 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:30.302406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 474/957] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:36:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:30.379130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b07160-d170-46c9-830b-0418dfe72484 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:30.492283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b07160-d170-46c9-830b-0418dfe72484 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:30.532929 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:30.533584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:30.533856 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:30.534118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:30.534692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:30.536038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:30.545763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:30.546875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:30.547759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 484/958] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:30.552858 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ddcb3-de9e-436d-af15-9f7eb3963678 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:30.643636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:30.658904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ddcb3-de9e-436d-af15-9f7eb3963678 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:30.697815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da2dd4-613c-41b4-807e-8fe1a837b15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:30.698443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2dd4-613c-41b4-807e-8fe1a837b15a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:30.698728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2dd4-613c-41b4-807e-8fe1a837b15a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:30.699003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da2dd4-613c-41b4-807e-8fe1a837b15a None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:30.699644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2dd4-613c-41b4-807e-8fe1a837b15a None admin] RBAC: Adding query filter params (name=compute) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:30.699948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2dd4-613c-41b4-807e-8fe1a837b15a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:30.701348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da2dd4-613c-41b4-807e-8fe1a837b15a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:30.707685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 475/959] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:30.712988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:30.738373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:30.824417 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c932dc5-5e50-452c-9bed-f6cc7845f643 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:30.861683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d55667e-e91a-49b2-844d-d676ca23b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:30.862653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55667e-e91a-49b2-844d-d676ca23b0c7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:30.863126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55667e-e91a-49b2-844d-d676ca23b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:30.863610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d55667e-e91a-49b2-844d-d676ca23b0c7 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:30.864427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55667e-e91a-49b2-844d-d676ca23b0c7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:30.864932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55667e-e91a-49b2-844d-d676ca23b0c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:30.866530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d55667e-e91a-49b2-844d-d676ca23b0c7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:30.871759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 485/960] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:31.204825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ac13-554c-4fcb-8b5e-a63a4b5213f7 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:31.205346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ac13-554c-4fcb-8b5e-a63a4b5213f7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:31.205346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353ac13-554c-4fcb-8b5e-a63a4b5213f7 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:31.206904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 476/961] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:31.212467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933cd93-6f20-4c19-ac77-5fa5d5573d15 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:31.212868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933cd93-6f20-4c19-ac77-5fa5d5573d15 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:31.213036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933cd93-6f20-4c19-ac77-5fa5d5573d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:31.232213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1933cd93-6f20-4c19-ac77-5fa5d5573d15 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:31.233488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1933cd93-6f20-4c19-ac77-5fa5d5573d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:31.303338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 486/962] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:31.582046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ac13-554c-4fcb-8b5e-a63a4b5213f7 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:31.687051 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353ac13-554c-4fcb-8b5e-a63a4b5213f7 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:31.721398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:31.721996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:31.722268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:31.722536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:31.723174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:31.724582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:31.739329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.sql.core [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 01:36:31.739329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 01:36:31.739329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [parameters: {'user_id': '262afa2ec78a43faa65c7ef28487d04e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 01:36:31.739329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87884) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application conn.query(q) Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result.read() Sep 09 01:36:31.748364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 01:36:31.749270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application next(self.gen) Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application yield resource Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application next(self.gen) Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise self.value Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application session.commit() Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 01:36:31.750842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.session.flush() Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application compat.raise_( Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 01:36:31.752349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 01:36:31.753473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application conn.query(q) Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application result.read() Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 01:36:31.754583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [parameters: {'user_id': '262afa2ec78a43faa65c7ef28487d04e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:31.755965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 01:36:31.757035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 01:36:31.758159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 01:36:31.758159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 01:36:31.758159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 01:36:31.758159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 01:36:31.758159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 01:36:31.758159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:31.758159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 477/963] 10.222.0.15 () {60 vars in 1131 bytes} [Fri Sep 9 01:36:31 2022] POST /identity/v3/users => generated 160 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:31.758574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933cd93-6f20-4c19-ac77-5fa5d5573d15 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:31.777082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:31.870559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933cd93-6f20-4c19-ac77-5fa5d5573d15 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:31.906737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5ad68-846c-4823-bdf9-2ae82e096ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:31.907571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ad68-846c-4823-bdf9-2ae82e096ef2 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:31.907961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ad68-846c-4823-bdf9-2ae82e096ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:31.908327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ad68-846c-4823-bdf9-2ae82e096ef2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:31.921127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5ad68-846c-4823-bdf9-2ae82e096ef2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:31.921127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5ad68-846c-4823-bdf9-2ae82e096ef2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-3ac5ad68-846c-4823-bdf9-2ae82e096ef2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:31.929296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:31.930371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:31.932419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 487/964] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:31.937663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:31.953785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:32.047026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfbed8-f68e-4b9e-8fba-912a62c28f7e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:32.082302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fdc98-05f9-4efc-a634-7bb76a79f6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:32.082918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fdc98-05f9-4efc-a634-7bb76a79f6be None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:32.083195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fdc98-05f9-4efc-a634-7bb76a79f6be None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:32.083454 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fdc98-05f9-4efc-a634-7bb76a79f6be None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:32.084158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fdc98-05f9-4efc-a634-7bb76a79f6be None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:32.084459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fdc98-05f9-4efc-a634-7bb76a79f6be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:32.086219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fdc98-05f9-4efc-a634-7bb76a79f6be None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:32.126174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 478/965] 10.222.0.15 () {56 vars in 1101 bytes} [Fri Sep 9 01:36:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:32.477147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c03a65-7c25-47b7-8db2-44b8bad04bba None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:32.477662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c03a65-7c25-47b7-8db2-44b8bad04bba None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:32.477662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c03a65-7c25-47b7-8db2-44b8bad04bba None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:32.479134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 488/966] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:32.484169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76da4-b070-42e5-a40c-8fdaa2a5fb27 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:32.484639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76da4-b070-42e5-a40c-8fdaa2a5fb27 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:32.484800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76da4-b070-42e5-a40c-8fdaa2a5fb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:32.503872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-57b76da4-b070-42e5-a40c-8fdaa2a5fb27 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:32.505710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-57b76da4-b070-42e5-a40c-8fdaa2a5fb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:32.573334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:32.579356 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 479/967] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:32.636105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950766-f4dc-4099-bcb9-14b352cb113e None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:32.636414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950766-f4dc-4099-bcb9-14b352cb113e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:32.636653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950766-f4dc-4099-bcb9-14b352cb113e None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:32.638146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 489/968] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:32.643306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e489c-baaf-4499-b3ee-0f2bf53bfa70 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:32.643590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e489c-baaf-4499-b3ee-0f2bf53bfa70 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:32.643864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e489c-baaf-4499-b3ee-0f2bf53bfa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:32.662058 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d15e489c-baaf-4499-b3ee-0f2bf53bfa70 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:32.663763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d15e489c-baaf-4499-b3ee-0f2bf53bfa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:32.730597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 480/969] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:32.844265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6950766-f4dc-4099-bcb9-14b352cb113e None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:32.928064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:32.956345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6950766-f4dc-4099-bcb9-14b352cb113e None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:32.992709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac6862-a315-45d5-94af-8b4c8a2094a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:32.993272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6862-a315-45d5-94af-8b4c8a2094a3 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:32.993547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6862-a315-45d5-94af-8b4c8a2094a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:32.993812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac6862-a315-45d5-94af-8b4c8a2094a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:32.999772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e489c-baaf-4499-b3ee-0f2bf53bfa70 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.004540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac6862-a315-45d5-94af-8b4c8a2094a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-a1ac6862-a315-45d5-94af-8b4c8a2094a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:33.009595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:33.010524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:33.011518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 490/970] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 01:36:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:33.016311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac6862-a315-45d5-94af-8b4c8a2094a3 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.102705 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e489c-baaf-4499-b3ee-0f2bf53bfa70 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.116265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac6862-a315-45d5-94af-8b4c8a2094a3 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.130783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:33.136823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90d342-436f-4aa9-9461-274964f78696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.137486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90d342-436f-4aa9-9461-274964f78696 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.137769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90d342-436f-4aa9-9461-274964f78696 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.138039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90d342-436f-4aa9-9461-274964f78696 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.138618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90d342-436f-4aa9-9461-274964f78696 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.139976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90d342-436f-4aa9-9461-274964f78696 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.149021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350d2c-4ac4-48a4-87b3-f7e11c21f8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-1a90d342-436f-4aa9-9461-274964f78696 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:33.149168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:33.150275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:33.151280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:33.152030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350d2c-4ac4-48a4-87b3-f7e11c21f8ee None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.152030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350d2c-4ac4-48a4-87b3-f7e11c21f8ee None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.152030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350d2c-4ac4-48a4-87b3-f7e11c21f8ee None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.152212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 481/971] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:33.152212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1a1e2-7083-4014-8538-972374c6dad0 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.152212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1a1e2-7083-4014-8538-972374c6dad0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.152212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1a1e2-7083-4014-8538-972374c6dad0 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.153285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350d2c-4ac4-48a4-87b3-f7e11c21f8ee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:33.153602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350d2c-4ac4-48a4-87b3-f7e11c21f8ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.154959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350d2c-4ac4-48a4-87b3-f7e11c21f8ee None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.156341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:33.157471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 482/972] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:33.158199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1a1e2-7083-4014-8538-972374c6dad0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.164556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 491/973] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:33.165764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb537-4961-4305-b14f-fc5588bd0623 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.166043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb537-4961-4305-b14f-fc5588bd0623 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.166311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb537-4961-4305-b14f-fc5588bd0623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.183626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7cbfb537-4961-4305-b14f-fc5588bd0623 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:33.184788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7cbfb537-4961-4305-b14f-fc5588bd0623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:33.249403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 492/974] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:33.250133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfb537-4961-4305-b14f-fc5588bd0623 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.268948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1a1e2-7083-4014-8538-972374c6dad0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.302059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.302656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.302930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.303196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.304332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:33.304633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.306006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.313215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 483/975] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:33.316024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.355070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfb537-4961-4305-b14f-fc5588bd0623 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.376367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:33.383307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:33.393205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.393829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.394104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.394368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.406097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.407518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.421315 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14fe3e-228d-4c17-8a97-85f179782288 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:33.423138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:33.424165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:33.425034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:33.425034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:33.425034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:33.425034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 493/976] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:33.429901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.456908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.457543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.457823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.458099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.458769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:33.459078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.460516 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.465170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 484/977] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:33.472343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.541630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968f4e5-af03-4e04-9909-4b9dad28c42e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.574260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.574863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.575140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.575409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.575585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd26df1-33f0-4bf6-a732-f34e4fcc218f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.576063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:33.576337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.577734 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.589753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 494/978] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:33.589753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.602744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:33.614643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.615248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.615522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.615791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.617029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] RBAC: Adding query filter params (interface=admin, service_id=f0917153d0c344209f2d3a20234036bd, region_id=RegionOne) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:33.617364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.618763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.623775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 485/979] 10.222.0.15 () {56 vars in 1249 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/endpoints?service_id=f0917153d0c344209f2d3a20234036bd&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:33.624553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.631023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:33.688846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aec11b-a672-4586-8f1c-83228a5a0ab9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.721984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94604c77-3b6e-4a95-8481-3c12f86db4d8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.722245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7069d-8358-432b-b001-c04983b95e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.722844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7069d-8358-432b-b001-c04983b95e56 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.723121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7069d-8358-432b-b001-c04983b95e56 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.723389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7069d-8358-432b-b001-c04983b95e56 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.724026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7069d-8358-432b-b001-c04983b95e56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.725401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7069d-8358-432b-b001-c04983b95e56 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.739362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 495/980] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:36:33 2022] POST /identity/v3/services => generated 242 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:33.740143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7069d-8358-432b-b001-c04983b95e56 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.767339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.768141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.768423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.768837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.778420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.780153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:33.793083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:33.794279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:33.795423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 486/981] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:33.800760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:33.851086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:33.851288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:33.858137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7069d-8358-432b-b001-c04983b95e56 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.903804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc9b9-6162-42b2-ab9f-33503f7f5add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.904428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc9b9-6162-42b2-ab9f-33503f7f5add None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.904706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc9b9-6162-42b2-ab9f-33503f7f5add None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.905477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc9b9-6162-42b2-ab9f-33503f7f5add None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.906215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fc9b9-6162-42b2-ab9f-33503f7f5add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.907459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fc9b9-6162-42b2-ab9f-33503f7f5add None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.915650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 496/982] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:36:33.926989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d43775-3e58-4a73-887a-3fde324b98a2 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:33.969271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:33.970163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:33.970603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:33.970868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:33.971617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:33.971950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:33.973440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:33.994905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 487/983] 10.222.0.15 () {56 vars in 1149 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:34.000431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc9b9-6162-42b2-ab9f-33503f7f5add None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:34.119454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc9b9-6162-42b2-ab9f-33503f7f5add None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:34.154556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea773449-fc39-49db-88b7-a4fc6e81c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:34.155172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea773449-fc39-49db-88b7-a4fc6e81c45b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:34.155448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea773449-fc39-49db-88b7-a4fc6e81c45b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:34.155720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea773449-fc39-49db-88b7-a4fc6e81c45b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:34.186636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:34.226559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea773449-fc39-49db-88b7-a4fc6e81c45b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:34.228047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea773449-fc39-49db-88b7-a4fc6e81c45b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:34.329026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 497/984] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:33 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:34.334449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:34.441770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:34.445302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfffa0c-0e67-4288-aff3-24dfbc5c3620 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:34.480941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:34.481693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:34.481976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:34.482244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:34.492776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:34.494244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:34.509194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:34.510150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:34.511026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:34.511026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:34.511026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:34.511809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 488/985] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:34.515886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea773449-fc39-49db-88b7-a4fc6e81c45b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:34.579045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:34.621727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea773449-fc39-49db-88b7-a4fc6e81c45b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:34.655757 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:34.656363 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:34.656638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:34.656905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:34.657569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:34.657871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:34.659226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:34.675336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 498/986] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:34.676644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:34.783417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6896ce-929c-4fbd-86fe-d356bf0ad6be None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:34.818560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:34.819193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:34.819505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:34.819775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:34.826717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:34.828138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:34.839648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:34.840509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:34.841933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:34.841933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:34.841933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:34.841933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:34.841933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:34.841933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:34.841933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:34.841933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:34.842651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 489/987] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:34.846657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:34.954278 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e566-5b44-4c1f-acf0-a6bf2002863f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:34.976170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:34.988456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411e69a-495e-490c-a54a-6ec991922fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:34.989071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411e69a-495e-490c-a54a-6ec991922fae None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:34.989395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411e69a-495e-490c-a54a-6ec991922fae None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:34.989670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411e69a-495e-490c-a54a-6ec991922fae None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:34.990349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411e69a-495e-490c-a54a-6ec991922fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:34.990650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411e69a-495e-490c-a54a-6ec991922fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:34.992088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411e69a-495e-490c-a54a-6ec991922fae None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:35.003924 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 499/988] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:35.010659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:35.026599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:35.111277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d03652-4d55-40e5-9c1c-513b00f7f63f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:35.148645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019df26-7ab1-4c98-aac2-d6d1d42af31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:35.149535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019df26-7ab1-4c98-aac2-d6d1d42af31a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:35.149938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019df26-7ab1-4c98-aac2-d6d1d42af31a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:35.150330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019df26-7ab1-4c98-aac2-d6d1d42af31a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:35.151420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019df26-7ab1-4c98-aac2-d6d1d42af31a None admin] RBAC: Adding query filter params (role.id=14e68dc1536843148f60370e1ea8013b, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=9cfe3dd0af154707bdaae5b52cbd6539) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:35.151878 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019df26-7ab1-4c98-aac2-d6d1d42af31a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:35.154078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019df26-7ab1-4c98-aac2-d6d1d42af31a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:35.169113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 490/989] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:35 2022] GET /identity/v3/role_assignments?user.id=9cfe3dd0af154707bdaae5b52cbd6539&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=14e68dc1536843148f60370e1ea8013b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:36.015707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1a92f-08cf-4451-a4f6-815ba8925142 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.016229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1a92f-08cf-4451-a4f6-815ba8925142 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.016229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1a92f-08cf-4451-a4f6-815ba8925142 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.017842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 500/990] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:36.023713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f5490-adf2-45b9-88d0-c97676471142 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.024231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f5490-adf2-45b9-88d0-c97676471142 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.024231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f5490-adf2-45b9-88d0-c97676471142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.043001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-633f5490-adf2-45b9-88d0-c97676471142 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:36.044189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-633f5490-adf2-45b9-88d0-c97676471142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:36.113023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 491/991] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:36.118685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474f10-63c5-4f78-8822-d8fb39f4624a None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.118972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474f10-63c5-4f78-8822-d8fb39f4624a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.119245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21474f10-63c5-4f78-8822-d8fb39f4624a None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.120693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 501/992] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:36.125650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9195f-993f-4c2f-a77d-3baf4df9e6ac None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.125943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9195f-993f-4c2f-a77d-3baf4df9e6ac None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.126220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9195f-993f-4c2f-a77d-3baf4df9e6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.139231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:36.145780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-5ff9195f-993f-4c2f-a77d-3baf4df9e6ac None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:36.147018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5ff9195f-993f-4c2f-a77d-3baf4df9e6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:36.213265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 492/993] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:36.379462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21474f10-63c5-4f78-8822-d8fb39f4624a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:36.480197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9195f-993f-4c2f-a77d-3baf4df9e6ac None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:36.480789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21474f10-63c5-4f78-8822-d8fb39f4624a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:36.523306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:36.523938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.524212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.524480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.525069 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:36.526450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:36.535462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:36.536528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:36.537402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:36.541211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 502/994] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:36.543019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:36.588615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9195f-993f-4c2f-a77d-3baf4df9e6ac None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:36.590308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:36.627045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:36.627664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.627935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.628313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.629047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:36.630702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:36.634231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:36.642263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:36.643247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:36.644657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 493/995] 10.222.0.15 () {56 vars in 1126 bytes} [Fri Sep 9 01:36:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:36.649931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:36.652006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de523c84-8c0a-4300-b764-feb355674394 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:36.687780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:36.688378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.688650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.688916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.689564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:36.690043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:36.691410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:36.701372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 503/996] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:36.702286 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:36.757828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0035738b-925c-48a8-8c01-0cc511c640ba None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:36.791560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:36.792165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.792443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.792713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.793367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:36.793674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:36.795032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:36.799960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 494/997] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:36.805460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:36.810080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6177ca-d0bd-4fed-967f-bfca83d06880 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:36.820640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:36.846459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:36.847071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.847352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.847624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.848247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:36.848772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:36.849927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:36.859872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 504/998] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:36.860851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:36.876637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:36.921301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caafa0-54db-43c1-abfa-25a942d47134 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:36.954874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:36.955473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:36.955757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:36.956008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:36.956628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:36.956923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:36.958303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:36.964069 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 495/999] 10.222.0.15 () {56 vars in 1122 bytes} [Fri Sep 9 01:36:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:36.969866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:36.976438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6907ec-ee46-46b9-a10e-3388d1875846 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:37.014043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:37.014658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:37.014934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:37.015202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:37.016205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] RBAC: Adding query filter params (interface=public, service_id=952877a42c804eb5b49299bb57372118, region_id=RegionOne) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:37.016205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:37.017596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:37.023587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 505/1000] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 01:36:36 2022] GET /identity/v3/endpoints?service_id=952877a42c804eb5b49299bb57372118&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:37.028995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:37.072706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:37.086632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c4ab2-f90b-4be1-b711-c32162b9bf11 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:37.118832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:37.127564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb5b86-dbae-44e1-b68b-4006bb87937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:37.128163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5b86-dbae-44e1-b68b-4006bb87937e None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:37.128444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5b86-dbae-44e1-b68b-4006bb87937e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:37.128715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5b86-dbae-44e1-b68b-4006bb87937e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:37.129399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb5b86-dbae-44e1-b68b-4006bb87937e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:37.131005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb5b86-dbae-44e1-b68b-4006bb87937e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:37.141276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2537b-2c82-4d35-9a9b-471b8e38ad5e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:37.161091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 496/1001] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:36:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:36:37.186035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ec8ea-f498-4596-b143-89da2d3a6f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:37.186650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ec8ea-f498-4596-b143-89da2d3a6f37 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:37.186921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ec8ea-f498-4596-b143-89da2d3a6f37 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:37.187186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ec8ea-f498-4596-b143-89da2d3a6f37 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:37.187792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ec8ea-f498-4596-b143-89da2d3a6f37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:37.189220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ec8ea-f498-4596-b143-89da2d3a6f37 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:37.201587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 506/1002] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:36:37 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:36:37.360399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38164-f82e-469a-a61c-11bfc8a24bc9 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:37.360744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38164-f82e-469a-a61c-11bfc8a24bc9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:37.360951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f38164-f82e-469a-a61c-11bfc8a24bc9 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:37.362374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 497/1003] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:37.369017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b27c0-29a9-4893-8b58-df85d8905c49 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:37.369351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b27c0-29a9-4893-8b58-df85d8905c49 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:37.369599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b27c0-29a9-4893-8b58-df85d8905c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:37.388695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b96b27c0-29a9-4893-8b58-df85d8905c49 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:37.389880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b96b27c0-29a9-4893-8b58-df85d8905c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:37.457917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 507/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:37.734556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f38164-f82e-469a-a61c-11bfc8a24bc9 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:37.841610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f38164-f82e-469a-a61c-11bfc8a24bc9 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:37.877179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:37.877822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:37.878103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:37.878374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] PATH_INFO: `/v3/services/image` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:37.878956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:37.880322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:37.890462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:37.891551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:37.892393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:37.893850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 498/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 9 01:36:37 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:37.900792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b27c0-29a9-4893-8b58-df85d8905c49 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:37.930949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:38.005194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b27c0-29a9-4893-8b58-df85d8905c49 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:38.039432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab164516-1964-48de-87e3-57ac4e8eb09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:38.040078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab164516-1964-48de-87e3-57ac4e8eb09f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:38.040300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab164516-1964-48de-87e3-57ac4e8eb09f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:38.040567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab164516-1964-48de-87e3-57ac4e8eb09f None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:38.041222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab164516-1964-48de-87e3-57ac4e8eb09f None admin] RBAC: Adding query filter params (name=image) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:38.041530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab164516-1964-48de-87e3-57ac4e8eb09f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:38.042882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab164516-1964-48de-87e3-57ac4e8eb09f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:38.052258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 508/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:38.053875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:38.113726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:38.163783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259e124-a69d-4cce-875c-82204e0264cc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:38.197288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec523136-d760-4084-bb47-5c814f1e5a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:38.197884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec523136-d760-4084-bb47-5c814f1e5a3b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:38.198161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec523136-d760-4084-bb47-5c814f1e5a3b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:38.198431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec523136-d760-4084-bb47-5c814f1e5a3b None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:38.199057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec523136-d760-4084-bb47-5c814f1e5a3b None admin] RBAC: Adding query filter params (type=image) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:38.199357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec523136-d760-4084-bb47-5c814f1e5a3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:38.200703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec523136-d760-4084-bb47-5c814f1e5a3b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:38.209163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 499/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:39.346089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49788aac-4045-4e7f-acbe-e8fc9bd3c5ad None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:39.346599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49788aac-4045-4e7f-acbe-e8fc9bd3c5ad None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:39.346704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49788aac-4045-4e7f-acbe-e8fc9bd3c5ad None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:39.348092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 509/1008] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:39.353471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aff61-709c-475f-aab8-818c4b865fbb None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:39.353925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aff61-709c-475f-aab8-818c4b865fbb None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:39.354007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aff61-709c-475f-aab8-818c4b865fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:39.373395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f81aff61-709c-475f-aab8-818c4b865fbb None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:39.374580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f81aff61-709c-475f-aab8-818c4b865fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:39.402470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0d0a-be02-4019-9ae9-cdc307d44ac9 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:39.403327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0d0a-be02-4019-9ae9-cdc307d44ac9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:39.403749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c0d0a-be02-4019-9ae9-cdc307d44ac9 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:39.405577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 510/1009] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:39.410671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619408b1-a21f-4b86-a94c-38e393824d39 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:39.411029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619408b1-a21f-4b86-a94c-38e393824d39 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:39.411377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619408b1-a21f-4b86-a94c-38e393824d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:39.430383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-619408b1-a21f-4b86-a94c-38e393824d39 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:39.431562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-619408b1-a21f-4b86-a94c-38e393824d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:39.457101 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 500/1010] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:39.500220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 511/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:39 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:39.721512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aff61-709c-475f-aab8-818c4b865fbb None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:39.768549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:39.773250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619408b1-a21f-4b86-a94c-38e393824d39 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:39.798489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:39.829017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aff61-709c-475f-aab8-818c4b865fbb None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:39.863823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fabd9a-9067-46f6-b25a-7a70f3167b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:39.864504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fabd9a-9067-46f6-b25a-7a70f3167b54 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:39.864783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fabd9a-9067-46f6-b25a-7a70f3167b54 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:39.865071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fabd9a-9067-46f6-b25a-7a70f3167b54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:39.866700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fabd9a-9067-46f6-b25a-7a70f3167b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:39.868111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fabd9a-9067-46f6-b25a-7a70f3167b54 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:39.869904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 501/1012] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:39.875547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fabd9a-9067-46f6-b25a-7a70f3167b54 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:39.885232 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619408b1-a21f-4b86-a94c-38e393824d39 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:39.927057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:39.927791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:39.928070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:39.928340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:39.929555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:39.931040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:39.943007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:39.944065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:39.945921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 512/1013] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:39.950491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:39.995723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fabd9a-9067-46f6-b25a-7a70f3167b54 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:40.010885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:40.032674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d5960-7de4-4ad0-bc20-dde33dcd925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:40.033317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d5960-7de4-4ad0-bc20-dde33dcd925d None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:40.033596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d5960-7de4-4ad0-bc20-dde33dcd925d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:40.033870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d5960-7de4-4ad0-bc20-dde33dcd925d None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:40.034515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d5960-7de4-4ad0-bc20-dde33dcd925d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:40.035922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d5960-7de4-4ad0-bc20-dde33dcd925d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:40.044482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:40.061426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e15ce2-cdd0-4e19-a4ab-ae7078947c8f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:40.063761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 502/1014] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:36:39 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:40.098089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:40.098715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:40.098991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:40.099262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:40.099903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] RBAC: Adding query filter params (name=compute) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:40.100203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:40.101749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:40.107259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 513/1015] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:40.113379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d5960-7de4-4ad0-bc20-dde33dcd925d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:40.239603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d5960-7de4-4ad0-bc20-dde33dcd925d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:40.308151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ee13c-f0e2-44ad-bed1-9652be075913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:40.308697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ee13c-f0e2-44ad-bed1-9652be075913 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:40.308980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ee13c-f0e2-44ad-bed1-9652be075913 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:40.309311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ee13c-f0e2-44ad-bed1-9652be075913 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:40.309954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ee13c-f0e2-44ad-bed1-9652be075913 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:40.310260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ee13c-f0e2-44ad-bed1-9652be075913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:40.311632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ee13c-f0e2-44ad-bed1-9652be075913 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:40.321139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 503/1016] 10.222.0.15 () {56 vars in 1112 bytes} [Fri Sep 9 01:36:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:40.325296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:40.360181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e0fbf-0a0c-42e3-ae63-cb57fab47e44 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:40.360708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e0fbf-0a0c-42e3-ae63-cb57fab47e44 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:40.360708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e0fbf-0a0c-42e3-ae63-cb57fab47e44 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:40.362169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 504/1017] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:40.367106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848d3c7-b2a3-40de-8d95-8046e5545588 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:40.367389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848d3c7-b2a3-40de-8d95-8046e5545588 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:40.367661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848d3c7-b2a3-40de-8d95-8046e5545588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:40.386907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4848d3c7-b2a3-40de-8d95-8046e5545588 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:40.388188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4848d3c7-b2a3-40de-8d95-8046e5545588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:40.413698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:40.454599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55cd20-66ca-4bf5-8bef-09a16c3afb5c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:40.467275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 505/1018] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:40 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:40.492082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b258b-ef88-4b53-a185-492b727c7532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:40.492797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b258b-ef88-4b53-a185-492b727c7532 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:40.493155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b258b-ef88-4b53-a185-492b727c7532 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:40.493487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b258b-ef88-4b53-a185-492b727c7532 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:40.494219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b258b-ef88-4b53-a185-492b727c7532 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:40.495671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b258b-ef88-4b53-a185-492b727c7532 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:40.518132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 514/1019] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:36:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:40.740143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848d3c7-b2a3-40de-8d95-8046e5545588 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:40.843910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848d3c7-b2a3-40de-8d95-8046e5545588 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:40.885785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94243197-593b-47eb-8562-618831132967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:40.886406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94243197-593b-47eb-8562-618831132967 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:40.886688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94243197-593b-47eb-8562-618831132967 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:40.886944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94243197-593b-47eb-8562-618831132967 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:40.887587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94243197-593b-47eb-8562-618831132967 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:40.888943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94243197-593b-47eb-8562-618831132967 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:40.913324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 506/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:36:40 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:42.345026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34ea6a-546f-428e-8a6c-c8c21c5dcffe None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:42.345554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34ea6a-546f-428e-8a6c-c8c21c5dcffe None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:42.345655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34ea6a-546f-428e-8a6c-c8c21c5dcffe None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:42.347078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 515/1021] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:42.352232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd127-1668-4f5a-a53b-7c13e97c84ab None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:42.352605 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd127-1668-4f5a-a53b-7c13e97c84ab None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:42.352784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bd127-1668-4f5a-a53b-7c13e97c84ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:42.371718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-5c9bd127-1668-4f5a-a53b-7c13e97c84ab None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:42.372902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5c9bd127-1668-4f5a-a53b-7c13e97c84ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:42.384431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:42.448874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 507/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:42.718418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34ea6a-546f-428e-8a6c-c8c21c5dcffe None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:42.727908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:42.810090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b01dd-145e-4323-8ec8-528f65b4260b None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:42.810368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b01dd-145e-4323-8ec8-528f65b4260b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:42.810754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b01dd-145e-4323-8ec8-528f65b4260b None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:42.812098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 508/1023] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:42.816897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad46453-dac5-487b-93c3-498e4304d1c0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:42.817205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad46453-dac5-487b-93c3-498e4304d1c0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:42.817478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad46453-dac5-487b-93c3-498e4304d1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:42.823959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34ea6a-546f-428e-8a6c-c8c21c5dcffe None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:42.836107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1ad46453-dac5-487b-93c3-498e4304d1c0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:42.837305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1ad46453-dac5-487b-93c3-498e4304d1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:42.859117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c16bc3-7f0e-4531-8c50-d684bcb64b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:42.859994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c16bc3-7f0e-4531-8c50-d684bcb64b17 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:42.860503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c16bc3-7f0e-4531-8c50-d684bcb64b17 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:42.860952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c16bc3-7f0e-4531-8c50-d684bcb64b17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:42.862751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c16bc3-7f0e-4531-8c50-d684bcb64b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:42.864351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c16bc3-7f0e-4531-8c50-d684bcb64b17 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:42.866173 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 516/1024] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:42.872092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c16bc3-7f0e-4531-8c50-d684bcb64b17 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:42.914216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 509/1025] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:42.956865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:42.983135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c16bc3-7f0e-4531-8c50-d684bcb64b17 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:43.021016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164eb6d-4df0-4952-ab70-ca95a67855af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:43.021656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164eb6d-4df0-4952-ab70-ca95a67855af None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.021934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164eb6d-4df0-4952-ab70-ca95a67855af None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.022203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164eb6d-4df0-4952-ab70-ca95a67855af None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.022851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164eb6d-4df0-4952-ab70-ca95a67855af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:43.024267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164eb6d-4df0-4952-ab70-ca95a67855af None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:43.053085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 517/1026] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:36:42 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:36:43.104649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba56bd7-c6b7-4579-8d70-2a896555c585 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.104968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba56bd7-c6b7-4579-8d70-2a896555c585 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.105249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba56bd7-c6b7-4579-8d70-2a896555c585 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.106640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 510/1027] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:43.112522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde8fbb-b310-44dd-b20f-bf95a6710a51 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.112833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde8fbb-b310-44dd-b20f-bf95a6710a51 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.113165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde8fbb-b310-44dd-b20f-bf95a6710a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.132532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1cde8fbb-b310-44dd-b20f-bf95a6710a51 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:43.133779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1cde8fbb-b310-44dd-b20f-bf95a6710a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:43.184618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba56bd7-c6b7-4579-8d70-2a896555c585 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.209453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 518/1028] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:43.267422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:43.302168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba56bd7-c6b7-4579-8d70-2a896555c585 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:43.338353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230c2e2-7578-4919-955b-ebdbe57ba769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:43.338981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c2e2-7578-4919-955b-ebdbe57ba769 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.339255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c2e2-7578-4919-955b-ebdbe57ba769 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.339523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c2e2-7578-4919-955b-ebdbe57ba769 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.349711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230c2e2-7578-4919-955b-ebdbe57ba769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-7230c2e2-7578-4919-955b-ebdbe57ba769 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:43.355073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:43.356394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:43.357659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 511/1029] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:43.362321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde8fbb-b310-44dd-b20f-bf95a6710a51 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.422559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:43.465789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde8fbb-b310-44dd-b20f-bf95a6710a51 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:43.481166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230c2e2-7578-4919-955b-ebdbe57ba769 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.500290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:43.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.501202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.501476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.502129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:43.502429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:43.503781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:43.517298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 519/1030] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:43.518529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.589579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230c2e2-7578-4919-955b-ebdbe57ba769 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:43.621338 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:43.627674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:43.628303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.628582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.628800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2590edc-ca09-4622-8a62-6cc5cb809d94 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:43.628917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] PATH_INFO: `/v3/services/image` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.629465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:43.631094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:43.640083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:43.645344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:43.646431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 512/1031] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:43.648812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.658643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:43.672713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:43.673380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.673662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.673933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.684719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:43.686183 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:43.700575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:43.701550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:43.707274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:43.707274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:43.707274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:43.707274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 520/1032] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:43.707274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.756329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb71157-306c-4794-95c8-55e4341b9bbe None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:43.791128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:43.792056 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.792876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.793433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.794582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] RBAC: Adding query filter params (name=image) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:43.795326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:43.797039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:43.802028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 513/1033] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:43.808101 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.821356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97c138-843f-4b53-bca7-35ae6d9c5173 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:43.857308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:43.857926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.858195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.858467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.859106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:43.859408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:43.860780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:43.871045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:43.878716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 521/1034] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:43.880694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.921149 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a3994-7ebd-4743-91dd-dab85dd6269e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:43.924729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:43.958454 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:43.958782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:43.958991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:43.959276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:43.959990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] RBAC: Adding query filter params (type=image) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:43.960295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:43.961679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:43.966548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 514/1035] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:43.972361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:43.996896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c0260-5648-423d-b3dc-bb92bc74909b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:44.043857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:44.044876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:44.045780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:44.045780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:44.052933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:44.054438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:44.066527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:44.067452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:44.068426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 522/1036] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:44.073774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:44.080017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccd7f1-1b57-438e-a421-67c7006eb90c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:44.104731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:44.116910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:44.117957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:44.118243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:44.118834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:44.119536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] RBAC: Adding query filter params (interface=public, service_id=63e3df4725a44c689e87a34a0be58cd3, region_id=RegionOne) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:44.119849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:44.121232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:44.131998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 515/1037] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 01:36:43 2022] GET /identity/v3/endpoints?service_id=63e3df4725a44c689e87a34a0be58cd3&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:44.132685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:44.171614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:44.179379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fe49d-62f2-496e-b1de-b26502263b9c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:44.215150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:44.215744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:44.216021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:44.216287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:44.216991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:44.217341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:44.218754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:44.236979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 523/1038] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:36:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:44.242629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:44.249383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2a854-bd0e-4388-a6be-2539ea00f340 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:44.284298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297fb23-b85b-44a3-ac5d-22f49636077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:44.284900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297fb23-b85b-44a3-ac5d-22f49636077e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:44.285212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297fb23-b85b-44a3-ac5d-22f49636077e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:44.285487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297fb23-b85b-44a3-ac5d-22f49636077e None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:44.286106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297fb23-b85b-44a3-ac5d-22f49636077e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:44.287479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297fb23-b85b-44a3-ac5d-22f49636077e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:44.305189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 516/1039] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:36:44 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:36:44.366180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baedf63-8e7c-4e3c-bb9b-c66cd94bf170 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:44.404302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2cbc7-42a6-4121-b020-ae1c81bf88c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:44.404925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2cbc7-42a6-4121-b020-ae1c81bf88c1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:44.405234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2cbc7-42a6-4121-b020-ae1c81bf88c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:44.405504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2cbc7-42a6-4121-b020-ae1c81bf88c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:44.429529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:44.481847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2cbc7-42a6-4121-b020-ae1c81bf88c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:44.483297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2cbc7-42a6-4121-b020-ae1c81bf88c1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:44.588394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 524/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:44 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:44.589650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297fb23-b85b-44a3-ac5d-22f49636077e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:44.643700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:44.694640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297fb23-b85b-44a3-ac5d-22f49636077e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:44.728362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:44.728989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:44.729293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:44.729563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:44.740195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:44.741697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:44.755945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:44.757678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:44.758600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:44.758600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:44.758600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:44.758600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 517/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:44.763366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2cbc7-42a6-4121-b020-ae1c81bf88c1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:44.820119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:44.869295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2cbc7-42a6-4121-b020-ae1c81bf88c1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:44.907229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:44.907833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:44.908109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:44.908375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:44.909031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:44.909355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:44.910695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:44.926684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 525/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:44.927972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:45.030595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30dfcad-27d3-4911-80ab-c0077e6a206d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:45.032958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:45.066434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:45.067064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:45.067341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:45.067611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:45.074146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:45.075572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:45.087268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:45.088160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:45.089151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 518/1043] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:45.094324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:45.199884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0be1db-488e-47c4-b3ce-587594a38dc5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:45.214217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:45.236365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf42bc79-35ab-4d85-be71-dd586af01b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:45.237180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42bc79-35ab-4d85-be71-dd586af01b6d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:45.237602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42bc79-35ab-4d85-be71-dd586af01b6d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:45.238044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf42bc79-35ab-4d85-be71-dd586af01b6d None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:45.238876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42bc79-35ab-4d85-be71-dd586af01b6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:45.239337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42bc79-35ab-4d85-be71-dd586af01b6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:45.241422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf42bc79-35ab-4d85-be71-dd586af01b6d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:45.254019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 526/1044] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:45.259991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:45.300068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48445c9d-a80c-40db-9bde-c0460f979a51 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:45.300388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48445c9d-a80c-40db-9bde-c0460f979a51 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:45.300614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48445c9d-a80c-40db-9bde-c0460f979a51 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:45.302086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 527/1045] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:45.306842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8286f75-7a3b-4566-b440-1e238cb586db None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:45.307118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8286f75-7a3b-4566-b440-1e238cb586db None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:45.307386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8286f75-7a3b-4566-b440-1e238cb586db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:45.325288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e8286f75-7a3b-4566-b440-1e238cb586db None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:45.326458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e8286f75-7a3b-4566-b440-1e238cb586db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:45.369169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e47a41-4378-4db1-b3e7-ad7eca1501eb None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:45.404830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 528/1046] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:45.407922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:45.408522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:45.408792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:45.409075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:45.409783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] RBAC: Adding query filter params (role.id=14e68dc1536843148f60370e1ea8013b, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:45.410085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:45.411491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:45.417639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 519/1047] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:45 2022] GET /identity/v3/role_assignments?user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=14e68dc1536843148f60370e1ea8013b => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:45.684764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8286f75-7a3b-4566-b440-1e238cb586db None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:45.784205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8286f75-7a3b-4566-b440-1e238cb586db None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:45.785744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:45.820413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cde9e6-4e95-4d94-a226-efa9b3b5001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:45.821045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cde9e6-4e95-4d94-a226-efa9b3b5001e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:45.821342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cde9e6-4e95-4d94-a226-efa9b3b5001e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:45.821611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cde9e6-4e95-4d94-a226-efa9b3b5001e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:45.831711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cde9e6-4e95-4d94-a226-efa9b3b5001e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-95cde9e6-4e95-4d94-a226-efa9b3b5001e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:45.836986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:45.837997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:45.838832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 529/1048] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:45.845515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:45.877344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:45.957145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a498d-ef10-479f-a270-a2501ac8ebcc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:45.992773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:45.993413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:45.993691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:45.993965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:45.994618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:45.994917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:45.996274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:46.005887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 520/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:46.012102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cde9e6-4e95-4d94-a226-efa9b3b5001e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:46.111384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cde9e6-4e95-4d94-a226-efa9b3b5001e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:46.145950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96aaaeb-2f0d-4e47-af99-ff7eda0862b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:46.146582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aaaeb-2f0d-4e47-af99-ff7eda0862b7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:46.146855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aaaeb-2f0d-4e47-af99-ff7eda0862b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:46.147160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96aaaeb-2f0d-4e47-af99-ff7eda0862b7 None admin] PATH_INFO: `/v3/users/e4dda5e6ac964350bec9b4ad0d5a63a7` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:46.160296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96aaaeb-2f0d-4e47-af99-ff7eda0862b7 None admin] RBAC: Authorizing `identity:get_user(user_id=e4dda5e6ac964350bec9b4ad0d5a63a7)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:46.161776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96aaaeb-2f0d-4e47-af99-ff7eda0862b7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:46.164119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 530/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:36:46 2022] GET /identity/v3/users/e4dda5e6ac964350bec9b4ad0d5a63a7 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:46.169606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:46.259603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:46.275518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59b676-d957-42bc-8580-d6bd66f91fe7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:46.312782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bc98f-3b53-4996-a1e3-604bb8cee05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:46.313417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc98f-3b53-4996-a1e3-604bb8cee05a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:46.313695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc98f-3b53-4996-a1e3-604bb8cee05a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:46.313965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc98f-3b53-4996-a1e3-604bb8cee05a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:46.384307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bc98f-3b53-4996-a1e3-604bb8cee05a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:46.385810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bc98f-3b53-4996-a1e3-604bb8cee05a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:46.473118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:46.477690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64920c1b-fec8-48ab-8233-83401013c794 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:46.478254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64920c1b-fec8-48ab-8233-83401013c794 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:46.478531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64920c1b-fec8-48ab-8233-83401013c794 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:46.480031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 531/1051] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:46.484673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638f16a-0af1-45d9-9137-02302dc78393 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:46.484953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638f16a-0af1-45d9-9137-02302dc78393 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:46.485249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638f16a-0af1-45d9-9137-02302dc78393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:46.492281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 521/1052] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:46.493654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bc98f-3b53-4996-a1e3-604bb8cee05a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:46.507229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2638f16a-0af1-45d9-9137-02302dc78393 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:46.508401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2638f16a-0af1-45d9-9137-02302dc78393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:46.522466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:46.583270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 532/1053] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:46.600077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bc98f-3b53-4996-a1e3-604bb8cee05a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:46.635688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a1a46-eb9c-4315-b637-7fb438556aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:46.636313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a1a46-eb9c-4315-b637-7fb438556aad None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:46.636589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a1a46-eb9c-4315-b637-7fb438556aad None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:46.636857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a1a46-eb9c-4315-b637-7fb438556aad None admin] PATH_INFO: `/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:46.637989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a1a46-eb9c-4315-b637-7fb438556aad None admin] RBAC: Authorizing `identity:get_project(project_id=17e361a47a7d4c9793f5f48253f4d3cb)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:46.639409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a1a46-eb9c-4315-b637-7fb438556aad None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:46.646236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 522/1054] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:36:46 2022] GET /identity/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:46.647138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638f16a-0af1-45d9-9137-02302dc78393 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:46.745327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638f16a-0af1-45d9-9137-02302dc78393 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:46.778702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:46.779234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:46.779509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:46.779779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:46.780491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=17e361a47a7d4c9793f5f48253f4d3cb, user.id=e4dda5e6ac964350bec9b4ad0d5a63a7) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:46.780795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:46.782236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:46.793117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 533/1055] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:46 2022] GET /identity/v3/role_assignments?user.id=e4dda5e6ac964350bec9b4ad0d5a63a7&scope.project.id=17e361a47a7d4c9793f5f48253f4d3cb&role.id=b131b61299154544a627315649090a49 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:46.853611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a1a46-eb9c-4315-b637-7fb438556aad None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:46.926121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:46.973455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a1a46-eb9c-4315-b637-7fb438556aad None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:47.008318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:47.008942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:47.009395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:47.009669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] PATH_INFO: `/v3/services/image` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:47.010247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:47.011597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:47.020203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:47.021442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 01:36:47.022336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:47.025142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 523/1056] 10.222.0.15 () {56 vars in 1105 bytes} [Fri Sep 9 01:36:46 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:47.028066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:47.044122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:47.128462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc9b6a-4219-4107-9aba-7d3a1fcb110c None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:47.162795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:47.163408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:47.163685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:47.163961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:47.164593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:47.164882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:47.166264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:47.171332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 534/1057] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:47.176967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:47.281484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970b9d9-bcdc-44f7-9f67-86ddf82decdc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:47.304202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:47.316439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a9846-caa4-4fbb-a14c-f5f1de86f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:47.317046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a9846-caa4-4fbb-a14c-f5f1de86f618 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:47.317347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a9846-caa4-4fbb-a14c-f5f1de86f618 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:47.317615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a9846-caa4-4fbb-a14c-f5f1de86f618 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:47.318239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a9846-caa4-4fbb-a14c-f5f1de86f618 None admin] RBAC: Adding query filter params (type=image) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:47.318539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a9846-caa4-4fbb-a14c-f5f1de86f618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:47.320506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a9846-caa4-4fbb-a14c-f5f1de86f618 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:47.325596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 524/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:47.331237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:47.410834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:47.434470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79428127-eadb-4f67-9625-9a2f4cc372e3 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:47.472463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4788cd-4f37-4e54-80ac-ccea89aced3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:47.473129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4788cd-4f37-4e54-80ac-ccea89aced3a None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:47.473409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4788cd-4f37-4e54-80ac-ccea89aced3a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:47.473680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4788cd-4f37-4e54-80ac-ccea89aced3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:47.474367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4788cd-4f37-4e54-80ac-ccea89aced3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:47.475793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4788cd-4f37-4e54-80ac-ccea89aced3a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:47.505085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 535/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:36:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:36:47.578340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a5f5-3ba5-41a6-bbab-0918a35a0947 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:47.578701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a5f5-3ba5-41a6-bbab-0918a35a0947 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:47.578903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4a5f5-3ba5-41a6-bbab-0918a35a0947 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:47.580515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 525/1060] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:47.586215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2088c2-6d07-4757-baa1-c36b97921ce5 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:47.586574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2088c2-6d07-4757-baa1-c36b97921ce5 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:47.586773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2088c2-6d07-4757-baa1-c36b97921ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:47.606809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-8f2088c2-6d07-4757-baa1-c36b97921ce5 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:47.608038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8f2088c2-6d07-4757-baa1-c36b97921ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:47.692173 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 536/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:47.960033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4a5f5-3ba5-41a6-bbab-0918a35a0947 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:48.065794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4a5f5-3ba5-41a6-bbab-0918a35a0947 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:48.099634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8373777-189a-4211-855b-60d5764f5370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:48.100847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373777-189a-4211-855b-60d5764f5370 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:48.101144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373777-189a-4211-855b-60d5764f5370 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:48.101412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8373777-189a-4211-855b-60d5764f5370 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:48.111754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8373777-189a-4211-855b-60d5764f5370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-f8373777-189a-4211-855b-60d5764f5370 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:48.116448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:48.117599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:48.118767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 526/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 01:36:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:48.125322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2088c2-6d07-4757-baa1-c36b97921ce5 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:48.183127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:48.226199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2088c2-6d07-4757-baa1-c36b97921ce5 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:48.262155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:48.262754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:48.263025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:48.263290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:48.263939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:48.264235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:48.265729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:48.278344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 537/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 01:36:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:48.279275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8373777-189a-4211-855b-60d5764f5370 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:48.325230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:48.378421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8373777-189a-4211-855b-60d5764f5370 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:48.411823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:48.412450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:48.413253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:48.413524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:48.424294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:48.425796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:48.442423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:48.443410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:48.444344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:48.444344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:48.444344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:48.448360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 527/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:48.449274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:48.547199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48924c81-a88c-421f-8ae1-ea9e220bbd57 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:48.568619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:48.581441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:48.582043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:48.582313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:48.582577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:48.583204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:48.583503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:48.585477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:48.597160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 538/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:48.602559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:48.698309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:48.702896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c83d3-cf98-4c96-aead-78d66b3f5dcc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:48.738827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62c8dd-113d-4f4e-b7a2-d7a1a2565961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:48.739450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62c8dd-113d-4f4e-b7a2-d7a1a2565961 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:48.739721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62c8dd-113d-4f4e-b7a2-d7a1a2565961 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:48.739990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62c8dd-113d-4f4e-b7a2-d7a1a2565961 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:48.747470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62c8dd-113d-4f4e-b7a2-d7a1a2565961 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:48.748905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62c8dd-113d-4f4e-b7a2-d7a1a2565961 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-ef62c8dd-113d-4f4e-b7a2-d7a1a2565961 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:48.761407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:48.762418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:48.763340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:48.765223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 528/1066] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:36:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:48.768215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:48.871787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b358b-7709-4492-95dd-a2ace4295c8f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:48.913132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:48.913752 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:48.914023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:48.914286 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:48.917337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:48.917337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:48.917337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:48.938558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 539/1067] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:36:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:48.939262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302fc48-72b8-4fad-98af-884c93f2940f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:48.939496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:48.939599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302fc48-72b8-4fad-98af-884c93f2940f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:48.939888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302fc48-72b8-4fad-98af-884c93f2940f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:48.941902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 529/1068] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:48.947245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e28cae-7fd2-4662-89aa-2fbfc03a4908 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:48.948094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e28cae-7fd2-4662-89aa-2fbfc03a4908 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:48.948628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e28cae-7fd2-4662-89aa-2fbfc03a4908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:48.971746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-32e28cae-7fd2-4662-89aa-2fbfc03a4908 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:48.972927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-32e28cae-7fd2-4662-89aa-2fbfc03a4908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:48.980895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:49.040955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebee99-d237-4ea8-8f2e-feea4fcdccad None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.056750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 530/1069] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:49.074516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.075136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.075409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.075674 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.085962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:49.087374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:49.102910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:49.103845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:49.104780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:49.104780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:49.104780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:49.104780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 540/1070] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:49.110099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e28cae-7fd2-4662-89aa-2fbfc03a4908 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:49.198293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:49.211493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e28cae-7fd2-4662-89aa-2fbfc03a4908 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.244806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.245442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.245717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.245985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.246612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:49.246962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:49.248334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:49.264787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 531/1071] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:49.266088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:49.335634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:49.358349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:49.369017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662acd1-df4d-4c90-ad15-6f2b8817631d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.403771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.404401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.404671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.404934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.412364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:49.414319 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:49.426172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:49.427039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:49.427927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:49.427927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:49.427927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:49.427927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:49.427927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:49.427927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:49.427927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:49.427927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:49.432471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 541/1072] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:49.433339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:49.441892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba04406-1681-467a-9c5d-123588abd89f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.475573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c6a38-e12b-4fbc-b74b-9539bd051ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.476123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c6a38-e12b-4fbc-b74b-9539bd051ebb None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.476403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c6a38-e12b-4fbc-b74b-9539bd051ebb None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.476778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c6a38-e12b-4fbc-b74b-9539bd051ebb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.487057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c6a38-e12b-4fbc-b74b-9539bd051ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-555c6a38-e12b-4fbc-b74b-9539bd051ebb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:49.491802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:49.492728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:49.493585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 532/1073] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:49.498842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c6a38-e12b-4fbc-b74b-9539bd051ebb None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:49.510281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:49.547567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b22b11-5d29-413c-bca1-b4dd407d5836 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.582178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.582785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.583063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.583336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.584016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:49.584846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:49.586301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:49.603896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 542/1074] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:49.605519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:49.607709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c6a38-e12b-4fbc-b74b-9539bd051ebb None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.624187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:49.646148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.647362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.647567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.647811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.648462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:49.649383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:49.650153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:49.659187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 533/1075] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:49.665265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:49.724204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4355e-4303-432f-81cc-13588757bb37 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.736614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:49.759586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e5dfd-eb2b-43f4-909e-f64d28f8da7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.760234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5dfd-eb2b-43f4-909e-f64d28f8da7b None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.760502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5dfd-eb2b-43f4-909e-f64d28f8da7b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.760774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e5dfd-eb2b-43f4-909e-f64d28f8da7b None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/5a63d93adb4e4044ae79fb5aa7d8f5e3/roles/14e68dc1536843148f60370e1ea8013b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.762923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e5dfd-eb2b-43f4-909e-f64d28f8da7b None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=5a63d93adb4e4044ae79fb5aa7d8f5e3, role_id=14e68dc1536843148f60370e1ea8013b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:49.764329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e5dfd-eb2b-43f4-909e-f64d28f8da7b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:49.769108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd4d99-de0f-4bde-ac58-909ff25578f1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.776966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 543/1076] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:36:49 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/5a63d93adb4e4044ae79fb5aa7d8f5e3/roles/14e68dc1536843148f60370e1ea8013b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:36:49.778154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432a449-b43f-4102-9db8-5a99d3911aa5 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.778434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432a449-b43f-4102-9db8-5a99d3911aa5 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.778701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432a449-b43f-4102-9db8-5a99d3911aa5 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.780096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 544/1077] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:49.785809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877b70a-187a-4f2c-a15a-45e8d1215b60 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.786115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877b70a-187a-4f2c-a15a-45e8d1215b60 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.786362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877b70a-187a-4f2c-a15a-45e8d1215b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.805681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6877b70a-187a-4f2c-a15a-45e8d1215b60 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:49.806911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6877b70a-187a-4f2c-a15a-45e8d1215b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:49.813375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8875a9a-d97c-4552-ab54-2119bca0e57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.814024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875a9a-d97c-4552-ab54-2119bca0e57e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.814301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875a9a-d97c-4552-ab54-2119bca0e57e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.814577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875a9a-d97c-4552-ab54-2119bca0e57e None admin] PATH_INFO: `/v3/users/e4dda5e6ac964350bec9b4ad0d5a63a7` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.815783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8875a9a-d97c-4552-ab54-2119bca0e57e None admin] RBAC: Authorizing `identity:get_user(user_id=e4dda5e6ac964350bec9b4ad0d5a63a7)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:49.817284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8875a9a-d97c-4552-ab54-2119bca0e57e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:49.819733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 534/1078] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:36:49 2022] GET /identity/v3/users/e4dda5e6ac964350bec9b4ad0d5a63a7 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:49.825408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8875a9a-d97c-4552-ab54-2119bca0e57e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:49.892669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 545/1079] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:49.948153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8875a9a-d97c-4552-ab54-2119bca0e57e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:49.988927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60671bed-c70f-4e95-881e-b4237d2a031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:49.989566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60671bed-c70f-4e95-881e-b4237d2a031e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:49.989785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60671bed-c70f-4e95-881e-b4237d2a031e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:49.990043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60671bed-c70f-4e95-881e-b4237d2a031e None admin] PATH_INFO: `/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:49.991712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60671bed-c70f-4e95-881e-b4237d2a031e None admin] RBAC: Authorizing `identity:get_project(project_id=17e361a47a7d4c9793f5f48253f4d3cb)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:49.993164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60671bed-c70f-4e95-881e-b4237d2a031e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:49.995011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 535/1080] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:36:49 2022] GET /identity/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:50.003236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877b70a-187a-4f2c-a15a-45e8d1215b60 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:50.006307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:50.104501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877b70a-187a-4f2c-a15a-45e8d1215b60 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:50.138952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fda8e-0f8b-44f6-881b-611e5dd568c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:50.139602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fda8e-0f8b-44f6-881b-611e5dd568c2 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:50.139874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fda8e-0f8b-44f6-881b-611e5dd568c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:50.140137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484fda8e-0f8b-44f6-881b-611e5dd568c2 None admin] PATH_INFO: `/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb/users/e4dda5e6ac964350bec9b4ad0d5a63a7/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:50.142190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fda8e-0f8b-44f6-881b-611e5dd568c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=17e361a47a7d4c9793f5f48253f4d3cb, user_id=e4dda5e6ac964350bec9b4ad0d5a63a7, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:50.143601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484fda8e-0f8b-44f6-881b-611e5dd568c2 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:50.156654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 546/1081] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:36:50 2022] PUT /identity/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb/users/e4dda5e6ac964350bec9b4ad0d5a63a7/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:36:50.188202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60671bed-c70f-4e95-881e-b4237d2a031e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:50.197591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:50.300160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60671bed-c70f-4e95-881e-b4237d2a031e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:50.339205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915b230-790d-4c25-a3d2-f4b18e494c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:50.339817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915b230-790d-4c25-a3d2-f4b18e494c14 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:50.340105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915b230-790d-4c25-a3d2-f4b18e494c14 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:50.340377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915b230-790d-4c25-a3d2-f4b18e494c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:50.410502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915b230-790d-4c25-a3d2-f4b18e494c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:50.411952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915b230-790d-4c25-a3d2-f4b18e494c14 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:50.420172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:50.512251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 536/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:50.518050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fda8e-0f8b-44f6-881b-611e5dd568c2 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:50.585443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:50.627379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484fda8e-0f8b-44f6-881b-611e5dd568c2 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:50.661652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019992d7-c1c1-48e8-918d-f84a599bd523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:50.662281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019992d7-c1c1-48e8-918d-f84a599bd523 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:50.663108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019992d7-c1c1-48e8-918d-f84a599bd523 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:50.663381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019992d7-c1c1-48e8-918d-f84a599bd523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:50.674851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019992d7-c1c1-48e8-918d-f84a599bd523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:50.676307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019992d7-c1c1-48e8-918d-f84a599bd523 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-019992d7-c1c1-48e8-918d-f84a599bd523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:50.692729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:50.694579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:50.696044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:50.696044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:50.696044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:50.697168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 547/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:50.703920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915b230-790d-4c25-a3d2-f4b18e494c14 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:50.811266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915b230-790d-4c25-a3d2-f4b18e494c14 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:50.819232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:50.848015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23176a9-8a64-4717-b645-1368065cb662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:50.848538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23176a9-8a64-4717-b645-1368065cb662 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:50.848811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23176a9-8a64-4717-b645-1368065cb662 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:50.849169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23176a9-8a64-4717-b645-1368065cb662 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:50.849799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23176a9-8a64-4717-b645-1368065cb662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:50.850627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23176a9-8a64-4717-b645-1368065cb662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:50.851971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23176a9-8a64-4717-b645-1368065cb662 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:50.868861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 537/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:51.980426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189839-cec7-4f5a-a5fa-d0b8ff1205c7 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:51.980963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189839-cec7-4f5a-a5fa-d0b8ff1205c7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:51.980963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189839-cec7-4f5a-a5fa-d0b8ff1205c7 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:51.982345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 548/1085] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:51.987853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e7dc2-a861-49ed-8868-184ec47b8c6f None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:51.988267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e7dc2-a861-49ed-8868-184ec47b8c6f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:51.988400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e7dc2-a861-49ed-8868-184ec47b8c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:52.007636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ff5e7dc2-a861-49ed-8868-184ec47b8c6f None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:52.008816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ff5e7dc2-a861-49ed-8868-184ec47b8c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:52.076702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 538/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:52.342225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79189839-cec7-4f5a-a5fa-d0b8ff1205c7 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:52.361080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd1573-2d2e-4108-b244-99da56300918 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:52.361376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd1573-2d2e-4108-b244-99da56300918 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:52.361648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd1573-2d2e-4108-b244-99da56300918 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:52.363004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 539/1087] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:36:52.368769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf75308-73f4-4cb1-9c94-4a0c3fb74fca None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:52.369045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf75308-73f4-4cb1-9c94-4a0c3fb74fca None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:52.369343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf75308-73f4-4cb1-9c94-4a0c3fb74fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:52.388687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ebf75308-73f4-4cb1-9c94-4a0c3fb74fca None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:52.389909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ebf75308-73f4-4cb1-9c94-4a0c3fb74fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:52.446092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79189839-cec7-4f5a-a5fa-d0b8ff1205c7 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:52.459927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:52.460358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 540/1088] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:36:52.479630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706011-718a-459d-8b6f-cf0975d000d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:52.480257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706011-718a-459d-8b6f-cf0975d000d9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:52.480532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706011-718a-459d-8b6f-cf0975d000d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:52.480798 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706011-718a-459d-8b6f-cf0975d000d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:52.490898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706011-718a-459d-8b6f-cf0975d000d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-be706011-718a-459d-8b6f-cf0975d000d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:52.495662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:52.496650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:52.497889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 549/1089] 10.222.0.15 () {56 vars in 1120 bytes} [Fri Sep 9 01:36:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:52.502591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf75308-73f4-4cb1-9c94-4a0c3fb74fca None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:52.529868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:52.599477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf75308-73f4-4cb1-9c94-4a0c3fb74fca None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:52.632071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:52.632658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:52.632931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:52.633233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:52.633881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:52.634227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:52.635567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:52.643130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 541/1090] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 01:36:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:52.649524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706011-718a-459d-8b6f-cf0975d000d9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:52.734127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:52.746632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706011-718a-459d-8b6f-cf0975d000d9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:52.779403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:52.780027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:52.780303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:52.780575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:52.791477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:52.792905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:52.807561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:52.808558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:52.809908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:52.809908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:52.809908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:52.809908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 550/1091] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:52.814253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:52.821477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:52.833489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb1aed-7729-46e1-946c-b2837b3f9085 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:52.842724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:52.867391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d2cbf-e2f1-4459-a5ff-e1cea68b0777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:52.868279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2cbf-e2f1-4459-a5ff-e1cea68b0777 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:52.868279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2cbf-e2f1-4459-a5ff-e1cea68b0777 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:52.868490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2cbf-e2f1-4459-a5ff-e1cea68b0777 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:52.878874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d2cbf-e2f1-4459-a5ff-e1cea68b0777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-9d3d2cbf-e2f1-4459-a5ff-e1cea68b0777 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:36:52.883997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:52.884947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:52.885821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 542/1092] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:52.891162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d2cbf-e2f1-4459-a5ff-e1cea68b0777 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:52.917963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4640b-04d0-4875-b1dd-ef4b83ff7884 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:52.952249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:52.952849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:52.953153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:52.953424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:52.954046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:52.954343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:52.955691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:52.967913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 551/1093] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:52.973459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:52.990026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d2cbf-e2f1-4459-a5ff-e1cea68b0777 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.023049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.023648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.023929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.024202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.024861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:53.025193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.026563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.034718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 543/1094] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:36:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:53.041405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:53.053787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:53.056414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:53.072281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44fcad-609a-4bae-a8fe-2033561225a7 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.106486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.107105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.107382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.107654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.114450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.116422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:36:53.128010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:53.129199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:53.130305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 552/1095] 10.222.0.15 () {56 vars in 1093 bytes} [Fri Sep 9 01:36:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:53.134987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:53.142826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a24489-7371-4113-9e1f-2809845925a4 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.176279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08248338-053a-4f6a-96d1-389ca5e262a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.176902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08248338-053a-4f6a-96d1-389ca5e262a8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.177213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08248338-053a-4f6a-96d1-389ca5e262a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.177486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08248338-053a-4f6a-96d1-389ca5e262a8 None admin] PATH_INFO: `/v3/users/e4dda5e6ac964350bec9b4ad0d5a63a7` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.178704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08248338-053a-4f6a-96d1-389ca5e262a8 None admin] RBAC: Authorizing `identity:get_user(user_id=e4dda5e6ac964350bec9b4ad0d5a63a7)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.180125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08248338-053a-4f6a-96d1-389ca5e262a8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.181736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 544/1096] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/users/e4dda5e6ac964350bec9b4ad0d5a63a7 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:53.187525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08248338-053a-4f6a-96d1-389ca5e262a8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:53.232546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ef0f4-e87b-430f-b2f0-e5628252d029 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.265668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:53.265975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.266686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.266997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.267303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.268070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:53.268411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.269971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.286860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 553/1097] 10.222.0.15 () {56 vars in 1133 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:53.288082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08248338-053a-4f6a-96d1-389ca5e262a8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.292878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:53.296285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:53.322789 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7819ce-a8cf-4e61-b03e-e247bad6f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.323381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7819ce-a8cf-4e61-b03e-e247bad6f85c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.323659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7819ce-a8cf-4e61-b03e-e247bad6f85c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.323930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7819ce-a8cf-4e61-b03e-e247bad6f85c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.389999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7819ce-a8cf-4e61-b03e-e247bad6f85c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.391400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7819ce-a8cf-4e61-b03e-e247bad6f85c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.392077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d03d4-c40f-4e97-9666-52d179901a2e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.425955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd362b-0e72-4085-a651-8806a3d4279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.426576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd362b-0e72-4085-a651-8806a3d4279b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.426852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd362b-0e72-4085-a651-8806a3d4279b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.427119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd362b-0e72-4085-a651-8806a3d4279b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.466419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:53.488844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 545/1098] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 01:36:53.493990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd362b-0e72-4085-a651-8806a3d4279b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.495427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd362b-0e72-4085-a651-8806a3d4279b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.495621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7819ce-a8cf-4e61-b03e-e247bad6f85c None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:53.498048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:53.595281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7819ce-a8cf-4e61-b03e-e247bad6f85c None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.595903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 554/1099] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 01:36:53.602211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd362b-0e72-4085-a651-8806a3d4279b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:53.628918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf14e57-902c-4907-b6b9-2879e0603ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.629579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf14e57-902c-4907-b6b9-2879e0603ddc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.629862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf14e57-902c-4907-b6b9-2879e0603ddc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.630144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf14e57-902c-4907-b6b9-2879e0603ddc None admin] PATH_INFO: `/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.631531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf14e57-902c-4907-b6b9-2879e0603ddc None admin] RBAC: Authorizing `identity:get_project(project_id=17e361a47a7d4c9793f5f48253f4d3cb)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.632949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf14e57-902c-4907-b6b9-2879e0603ddc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.634607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 546/1100] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:53.640607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf14e57-902c-4907-b6b9-2879e0603ddc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:53.689716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:53.706080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd362b-0e72-4085-a651-8806a3d4279b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.708521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:53.740831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.741480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.741753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.742015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.742483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf14e57-902c-4907-b6b9-2879e0603ddc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.751806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.753267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:36:53.769012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:53.769963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:36:53.771457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:36:53.771457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:36:53.771457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:36:53.771457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 555/1101] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 01:36:53.776407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:53.778486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.779086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.779363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.779633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.780349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=17e361a47a7d4c9793f5f48253f4d3cb, user.id=e4dda5e6ac964350bec9b4ad0d5a63a7) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:53.780654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.782095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.788804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 547/1102] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/role_assignments?user.id=e4dda5e6ac964350bec9b4ad0d5a63a7&scope.project.id=17e361a47a7d4c9793f5f48253f4d3cb&role.id=b131b61299154544a627315649090a49 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:53.879855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae43a3-7004-47fb-8792-9e7a45688d4e None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:53.918075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:53.918764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:53.919040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:53.919331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:53.920000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:53.920306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:53.921752 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:53.934820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 556/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:53.940628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:54.040199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29fc0ad-1127-453a-a92f-2a3344b207cc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:54.050203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:54.074096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:54.074712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:54.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:54.075250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:54.081664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:54.083067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:36:54.093909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:36:54.094846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:36:54.096260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 548/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:36:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:36:54.100811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:54.119310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:54.198890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbccfb-d134-4e1b-94a9-ff3ab4a8937b None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:54.231332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e517e-d039-460f-abbe-05bef81c6801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:54.231960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e517e-d039-460f-abbe-05bef81c6801 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:54.232308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e517e-d039-460f-abbe-05bef81c6801 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:54.232570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e517e-d039-460f-abbe-05bef81c6801 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:54.233277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e517e-d039-460f-abbe-05bef81c6801 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:54.233613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e517e-d039-460f-abbe-05bef81c6801 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:54.235019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e517e-d039-460f-abbe-05bef81c6801 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:54.246849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 557/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:36:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:36:54.253121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:54.350592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6a4d1-e612-45d0-864b-39cbcf280881 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:54.384072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58233002-22e9-42fc-8d61-d04144dfedb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:54.389494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58233002-22e9-42fc-8d61-d04144dfedb0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:54.389494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58233002-22e9-42fc-8d61-d04144dfedb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:54.389494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58233002-22e9-42fc-8d61-d04144dfedb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:54.389494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58233002-22e9-42fc-8d61-d04144dfedb0 None admin] RBAC: Adding query filter params (role.id=14e68dc1536843148f60370e1ea8013b, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:54.389494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58233002-22e9-42fc-8d61-d04144dfedb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:54.390256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58233002-22e9-42fc-8d61-d04144dfedb0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:54.396043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 549/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:54 2022] GET /identity/v3/role_assignments?user.id=5a63d93adb4e4044ae79fb5aa7d8f5e3&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=14e68dc1536843148f60370e1ea8013b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:55.960405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da950-8931-43c4-8d4a-bdfd8b555111 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:55.960946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da950-8931-43c4-8d4a-bdfd8b555111 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:55.960946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6da950-8931-43c4-8d4a-bdfd8b555111 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:55.962299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 558/1107] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:55.967599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e583db3-11e8-4334-9680-7736d20408d2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:55.967996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e583db3-11e8-4334-9680-7736d20408d2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:55.968155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e583db3-11e8-4334-9680-7736d20408d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:55.985928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9e583db3-11e8-4334-9680-7736d20408d2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:55.987119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9e583db3-11e8-4334-9680-7736d20408d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:56.013268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:56.053478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 550/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:56.321476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6da950-8931-43c4-8d4a-bdfd8b555111 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:56.397416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:56.421435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6da950-8931-43c4-8d4a-bdfd8b555111 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:56.452977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5d25d-1098-4861-84d1-5760646c2c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:56.453643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5d25d-1098-4861-84d1-5760646c2c73 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:56.453913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5d25d-1098-4861-84d1-5760646c2c73 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:56.454176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5d25d-1098-4861-84d1-5760646c2c73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:56.455261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5d25d-1098-4861-84d1-5760646c2c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:56.456663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5d25d-1098-4861-84d1-5760646c2c73 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:56.458237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 559/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:56.464468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e583db3-11e8-4334-9680-7736d20408d2 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:56.558795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e583db3-11e8-4334-9680-7736d20408d2 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:56.591167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a955-8617-4074-a3a0-cb2adc54cd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:56.591701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a955-8617-4074-a3a0-cb2adc54cd3f None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:56.591979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a955-8617-4074-a3a0-cb2adc54cd3f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:56.592249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a955-8617-4074-a3a0-cb2adc54cd3f None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:56.592895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a955-8617-4074-a3a0-cb2adc54cd3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:56.594329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a955-8617-4074-a3a0-cb2adc54cd3f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:56.615221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 551/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:36:56 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:36:58.776693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145306a-b13c-41cf-ad0c-dbe539f44f26 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:58.777277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145306a-b13c-41cf-ad0c-dbe539f44f26 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:58.777277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145306a-b13c-41cf-ad0c-dbe539f44f26 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:58.778570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 560/1111] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:36:58.783846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaf2e0-a2ce-4c65-8ffc-26af652a2be9 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:58.784240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaf2e0-a2ce-4c65-8ffc-26af652a2be9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:58.784399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aaf2e0-a2ce-4c65-8ffc-26af652a2be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:58.802793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b8aaf2e0-a2ce-4c65-8ffc-26af652a2be9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:36:58.804024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b8aaf2e0-a2ce-4c65-8ffc-26af652a2be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:36:58.817485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:58.869037 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 552/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:36:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:36:59.135979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145306a-b13c-41cf-ad0c-dbe539f44f26 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:59.233018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145306a-b13c-41cf-ad0c-dbe539f44f26 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:59.268572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aba6b5-84d5-411c-993c-95a16640a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:59.269217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba6b5-84d5-411c-993c-95a16640a006 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:59.269493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba6b5-84d5-411c-993c-95a16640a006 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:59.269759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aba6b5-84d5-411c-993c-95a16640a006 None admin] PATH_INFO: `/v3/roles/eb7bedf01e0744eeb26681978db4121c` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:59.271117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aba6b5-84d5-411c-993c-95a16640a006 None admin] RBAC: Authorizing `identity:get_role(role_id=eb7bedf01e0744eeb26681978db4121c)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:59.273109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aba6b5-84d5-411c-993c-95a16640a006 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:59.274404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 561/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:36:59 2022] GET /identity/v3/roles/eb7bedf01e0744eeb26681978db4121c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:59.280225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaf2e0-a2ce-4c65-8ffc-26af652a2be9 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:59.373855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aaf2e0-a2ce-4c65-8ffc-26af652a2be9 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:59.406976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc12596-e8e1-4255-9dae-1dab0c46071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:59.407527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12596-e8e1-4255-9dae-1dab0c46071e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:59.407799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12596-e8e1-4255-9dae-1dab0c46071e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:59.408062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc12596-e8e1-4255-9dae-1dab0c46071e None admin] PATH_INFO: `/v3/users/f10e45c44f984780bed3c862e5c3d8e6` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:59.420899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc12596-e8e1-4255-9dae-1dab0c46071e None admin] RBAC: Authorizing `identity:get_user(user_id=f10e45c44f984780bed3c862e5c3d8e6)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:59.422330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc12596-e8e1-4255-9dae-1dab0c46071e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:59.423882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 553/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:36:59 2022] GET /identity/v3/users/f10e45c44f984780bed3c862e5c3d8e6 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:59.429930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aba6b5-84d5-411c-993c-95a16640a006 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:59.445777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:59.526832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aba6b5-84d5-411c-993c-95a16640a006 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:59.558476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108939-f490-453d-9ff0-ce1a004bd333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:59.559061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108939-f490-453d-9ff0-ce1a004bd333 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:59.559336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108939-f490-453d-9ff0-ce1a004bd333 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:59.559605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108939-f490-453d-9ff0-ce1a004bd333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:59.622866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40108939-f490-453d-9ff0-ce1a004bd333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:59.624289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40108939-f490-453d-9ff0-ce1a004bd333 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:59.637693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:59.718033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 562/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:36:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:36:59.723787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc12596-e8e1-4255-9dae-1dab0c46071e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:59.734740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:59.820794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc12596-e8e1-4255-9dae-1dab0c46071e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:59.853179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0212390-9d05-49c0-bf2b-ebdf12b72e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:59.853791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0212390-9d05-49c0-bf2b-ebdf12b72e37 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:59.854068 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0212390-9d05-49c0-bf2b-ebdf12b72e37 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:59.854334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0212390-9d05-49c0-bf2b-ebdf12b72e37 None admin] PATH_INFO: `/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:59.855436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0212390-9d05-49c0-bf2b-ebdf12b72e37 None admin] RBAC: Authorizing `identity:get_project(project_id=17e361a47a7d4c9793f5f48253f4d3cb)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:59.856850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0212390-9d05-49c0-bf2b-ebdf12b72e37 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:36:59.858492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 554/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:36:59 2022] GET /identity/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:36:59.864443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108939-f490-453d-9ff0-ce1a004bd333 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:36:59.961627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108939-f490-453d-9ff0-ce1a004bd333 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:36:59.974244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:36:59.994496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337290b9-b644-4ff5-9cf3-fb54d35172fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:36:59.995088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337290b9-b644-4ff5-9cf3-fb54d35172fe None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:36:59.995362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337290b9-b644-4ff5-9cf3-fb54d35172fe None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:36:59.995629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337290b9-b644-4ff5-9cf3-fb54d35172fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:36:59.996337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337290b9-b644-4ff5-9cf3-fb54d35172fe None admin] RBAC: Adding query filter params (role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=17e361a47a7d4c9793f5f48253f4d3cb, user.id=f10e45c44f984780bed3c862e5c3d8e6) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:36:59.996640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337290b9-b644-4ff5-9cf3-fb54d35172fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:36:59.998069 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337290b9-b644-4ff5-9cf3-fb54d35172fe None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:00.003271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 563/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:36:59 2022] GET /identity/v3/role_assignments?user.id=f10e45c44f984780bed3c862e5c3d8e6&scope.project.id=17e361a47a7d4c9793f5f48253f4d3cb&role.id=eb7bedf01e0744eeb26681978db4121c => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:02.085911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331f3b4-58fc-4e86-b897-de3c09052473 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:02.086460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331f3b4-58fc-4e86-b897-de3c09052473 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:02.086460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331f3b4-58fc-4e86-b897-de3c09052473 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:02.087811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 555/1118] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:02.092815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76e57-377d-486f-aef4-84bbb8f5aae6 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:02.093268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76e57-377d-486f-aef4-84bbb8f5aae6 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:02.093393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76e57-377d-486f-aef4-84bbb8f5aae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:02.111578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1ed76e57-377d-486f-aef4-84bbb8f5aae6 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:02.112761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1ed76e57-377d-486f-aef4-84bbb8f5aae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:02.176828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 564/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:02.440428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331f3b4-58fc-4e86-b897-de3c09052473 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:02.516937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:02.540175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331f3b4-58fc-4e86-b897-de3c09052473 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:02.572715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd07fe-f8dd-40bb-b74f-3feda478e1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:02.573356 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd07fe-f8dd-40bb-b74f-3feda478e1a1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:02.573637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd07fe-f8dd-40bb-b74f-3feda478e1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:02.573907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd07fe-f8dd-40bb-b74f-3feda478e1a1 None admin] PATH_INFO: `/v3/roles/eb7bedf01e0744eeb26681978db4121c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:02.575277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd07fe-f8dd-40bb-b74f-3feda478e1a1 None admin] RBAC: Authorizing `identity:get_role(role_id=eb7bedf01e0744eeb26681978db4121c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:02.576634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd07fe-f8dd-40bb-b74f-3feda478e1a1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:02.577963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 556/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:02 2022] GET /identity/v3/roles/eb7bedf01e0744eeb26681978db4121c => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:02.583614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed76e57-377d-486f-aef4-84bbb8f5aae6 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:02.678816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed76e57-377d-486f-aef4-84bbb8f5aae6 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:02.685751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:02.712189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ada60-01aa-4bbb-b876-62a8442468dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:02.712755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ada60-01aa-4bbb-b876-62a8442468dd None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:02.713032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ada60-01aa-4bbb-b876-62a8442468dd None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:02.713339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ada60-01aa-4bbb-b876-62a8442468dd None admin] PATH_INFO: `/v3/users/f10e45c44f984780bed3c862e5c3d8e6` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:02.714467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ada60-01aa-4bbb-b876-62a8442468dd None admin] RBAC: Authorizing `identity:get_user(user_id=f10e45c44f984780bed3c862e5c3d8e6)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:02.715874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ada60-01aa-4bbb-b876-62a8442468dd None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:02.717454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 565/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:02 2022] GET /identity/v3/users/f10e45c44f984780bed3c862e5c3d8e6 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:02.723108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd07fe-f8dd-40bb-b74f-3feda478e1a1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:02.820495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd07fe-f8dd-40bb-b74f-3feda478e1a1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:02.853359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cd198-37b9-4f98-bd34-0bfa525acdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:02.853969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cd198-37b9-4f98-bd34-0bfa525acdd3 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:02.854248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cd198-37b9-4f98-bd34-0bfa525acdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:02.854521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cd198-37b9-4f98-bd34-0bfa525acdd3 None admin] PATH_INFO: `/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:02.855619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cd198-37b9-4f98-bd34-0bfa525acdd3 None admin] RBAC: Authorizing `identity:get_project(project_id=17e361a47a7d4c9793f5f48253f4d3cb)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:02.857017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cd198-37b9-4f98-bd34-0bfa525acdd3 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:02.858637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 557/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:37:02 2022] GET /identity/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:02.864350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ada60-01aa-4bbb-b876-62a8442468dd None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:02.960534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ada60-01aa-4bbb-b876-62a8442468dd None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:02.994194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56324e-549d-4928-9b21-cbb1854c12b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:02.994779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56324e-549d-4928-9b21-cbb1854c12b3 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:02.995056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56324e-549d-4928-9b21-cbb1854c12b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:02.995329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56324e-549d-4928-9b21-cbb1854c12b3 None admin] PATH_INFO: `/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb/users/f10e45c44f984780bed3c862e5c3d8e6/roles/eb7bedf01e0744eeb26681978db4121c` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:02.997470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56324e-549d-4928-9b21-cbb1854c12b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=17e361a47a7d4c9793f5f48253f4d3cb, user_id=f10e45c44f984780bed3c862e5c3d8e6, role_id=eb7bedf01e0744eeb26681978db4121c)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:02.998881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56324e-549d-4928-9b21-cbb1854c12b3 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:03.011284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 566/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:37:02 2022] PUT /identity/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb/users/f10e45c44f984780bed3c862e5c3d8e6/roles/eb7bedf01e0744eeb26681978db4121c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:37:05.125753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1e871-2257-4678-abda-cfd37a5be2c3 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:05.126301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1e871-2257-4678-abda-cfd37a5be2c3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:05.126301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1e871-2257-4678-abda-cfd37a5be2c3 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:05.127612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 558/1124] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:05.133621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349dade-f28e-47bb-a630-b63270d0fbdd None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:05.134004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349dade-f28e-47bb-a630-b63270d0fbdd None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:05.134169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349dade-f28e-47bb-a630-b63270d0fbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:05.152771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7349dade-f28e-47bb-a630-b63270d0fbdd None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:05.153922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7349dade-f28e-47bb-a630-b63270d0fbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:05.179120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:05.218844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 567/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:05.492873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1e871-2257-4678-abda-cfd37a5be2c3 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:05.508234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:05.590735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1e871-2257-4678-abda-cfd37a5be2c3 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:05.623139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1622b-44e8-45bd-b888-25b89b6a8f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:05.623757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1622b-44e8-45bd-b888-25b89b6a8f23 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:05.624037 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1622b-44e8-45bd-b888-25b89b6a8f23 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:05.624308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1622b-44e8-45bd-b888-25b89b6a8f23 None admin] PATH_INFO: `/v3/roles/eb7bedf01e0744eeb26681978db4121c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:05.625711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1622b-44e8-45bd-b888-25b89b6a8f23 None admin] RBAC: Authorizing `identity:get_role(role_id=eb7bedf01e0744eeb26681978db4121c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:05.627075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1622b-44e8-45bd-b888-25b89b6a8f23 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:05.628383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 559/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:05 2022] GET /identity/v3/roles/eb7bedf01e0744eeb26681978db4121c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:05.634810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349dade-f28e-47bb-a630-b63270d0fbdd None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:05.732120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349dade-f28e-47bb-a630-b63270d0fbdd None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:05.770344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef92cf-85f9-4dbc-b7c4-85de480539c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:05.770889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef92cf-85f9-4dbc-b7c4-85de480539c1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:05.771162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef92cf-85f9-4dbc-b7c4-85de480539c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:05.771430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef92cf-85f9-4dbc-b7c4-85de480539c1 None admin] PATH_INFO: `/v3/users/f10e45c44f984780bed3c862e5c3d8e6` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:05.772530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef92cf-85f9-4dbc-b7c4-85de480539c1 None admin] RBAC: Authorizing `identity:get_user(user_id=f10e45c44f984780bed3c862e5c3d8e6)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:05.773982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef92cf-85f9-4dbc-b7c4-85de480539c1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:05.775545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 568/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:05 2022] GET /identity/v3/users/f10e45c44f984780bed3c862e5c3d8e6 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:05.781380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1622b-44e8-45bd-b888-25b89b6a8f23 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:05.858666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:05.877746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1622b-44e8-45bd-b888-25b89b6a8f23 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:05.910598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee42413-9af1-4d81-a190-5509658b00a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:05.911189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee42413-9af1-4d81-a190-5509658b00a0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:05.911461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee42413-9af1-4d81-a190-5509658b00a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:05.911727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee42413-9af1-4d81-a190-5509658b00a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:05.976376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee42413-9af1-4d81-a190-5509658b00a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:05.977829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee42413-9af1-4d81-a190-5509658b00a0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:06.053456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:06.072951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 560/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:37:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:37:06.079006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef92cf-85f9-4dbc-b7c4-85de480539c1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:06.115226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:06.184220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef92cf-85f9-4dbc-b7c4-85de480539c1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:06.219647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419cb3d-ecd8-4e71-b5d1-d7493c27f926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:06.220260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419cb3d-ecd8-4e71-b5d1-d7493c27f926 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:06.220540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419cb3d-ecd8-4e71-b5d1-d7493c27f926 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:06.220810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419cb3d-ecd8-4e71-b5d1-d7493c27f926 None admin] PATH_INFO: `/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:06.221945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419cb3d-ecd8-4e71-b5d1-d7493c27f926 None admin] RBAC: Authorizing `identity:get_project(project_id=17e361a47a7d4c9793f5f48253f4d3cb)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:06.223361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419cb3d-ecd8-4e71-b5d1-d7493c27f926 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:06.225601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 569/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:37:06 2022] GET /identity/v3/projects/17e361a47a7d4c9793f5f48253f4d3cb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:06.231774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee42413-9af1-4d81-a190-5509658b00a0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:06.326096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee42413-9af1-4d81-a190-5509658b00a0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:06.359473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aafa1e-5c33-4adf-a999-f268c9a732c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:06.360053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aafa1e-5c33-4adf-a999-f268c9a732c4 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:06.360324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aafa1e-5c33-4adf-a999-f268c9a732c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:06.360588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aafa1e-5c33-4adf-a999-f268c9a732c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:06.361324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aafa1e-5c33-4adf-a999-f268c9a732c4 None admin] RBAC: Adding query filter params (role.id=eb7bedf01e0744eeb26681978db4121c, scope.project.id=17e361a47a7d4c9793f5f48253f4d3cb, user.id=f10e45c44f984780bed3c862e5c3d8e6) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:06.361624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aafa1e-5c33-4adf-a999-f268c9a732c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:06.363018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aafa1e-5c33-4adf-a999-f268c9a732c4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:06.368484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 561/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:37:06 2022] GET /identity/v3/role_assignments?user.id=f10e45c44f984780bed3c862e5c3d8e6&scope.project.id=17e361a47a7d4c9793f5f48253f4d3cb&role.id=eb7bedf01e0744eeb26681978db4121c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:08.523800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae432f-d888-456d-88d8-1cb90b379327 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:08.524351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae432f-d888-456d-88d8-1cb90b379327 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:08.524351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae432f-d888-456d-88d8-1cb90b379327 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:08.525695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 570/1131] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:08.531030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b84fd-e3f5-453a-90c2-409eb2a647fe None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:08.531448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b84fd-e3f5-453a-90c2-409eb2a647fe None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:08.531570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b84fd-e3f5-453a-90c2-409eb2a647fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:08.549664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0d7b84fd-e3f5-453a-90c2-409eb2a647fe None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:08.550819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0d7b84fd-e3f5-453a-90c2-409eb2a647fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:08.588598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:08.615220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 562/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:08.888019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae432f-d888-456d-88d8-1cb90b379327 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:08.981747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:08.985889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae432f-d888-456d-88d8-1cb90b379327 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:09.018105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb574393-a241-440c-b634-5c70c68ef4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:09.018713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574393-a241-440c-b634-5c70c68ef4ea None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:09.018987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574393-a241-440c-b634-5c70c68ef4ea None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:09.019253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb574393-a241-440c-b634-5c70c68ef4ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:09.020338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb574393-a241-440c-b634-5c70c68ef4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:09.021793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb574393-a241-440c-b634-5c70c68ef4ea None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:09.023320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 571/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:37:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:09.029030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b84fd-e3f5-453a-90c2-409eb2a647fe None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:09.123081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b84fd-e3f5-453a-90c2-409eb2a647fe None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:09.155357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666ad36-75cd-464c-b90d-4bf3a9f99d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:09.155887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666ad36-75cd-464c-b90d-4bf3a9f99d8b None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:09.156161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666ad36-75cd-464c-b90d-4bf3a9f99d8b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:09.156425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666ad36-75cd-464c-b90d-4bf3a9f99d8b None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:09.157085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666ad36-75cd-464c-b90d-4bf3a9f99d8b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:09.158489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666ad36-75cd-464c-b90d-4bf3a9f99d8b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:09.184905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 563/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:37:09 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:37:11.343964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e937-25c1-4183-ae94-91d33b4154db None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:11.344510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e937-25c1-4183-ae94-91d33b4154db None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:11.344510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e937-25c1-4183-ae94-91d33b4154db None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:11.345871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 572/1135] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:11.351125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd472ea-e1ab-400f-8652-0b84b83e497d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:11.351511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd472ea-e1ab-400f-8652-0b84b83e497d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:11.351675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd472ea-e1ab-400f-8652-0b84b83e497d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:11.369955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-bdd472ea-e1ab-400f-8652-0b84b83e497d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:11.371118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bdd472ea-e1ab-400f-8652-0b84b83e497d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:11.394187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:11.439364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 564/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:11.712205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e937-25c1-4183-ae94-91d33b4154db None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:11.815539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e937-25c1-4183-ae94-91d33b4154db None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:11.847243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2fb3b7-6bb3-4b2c-a96b-5d11cfbb2a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:11.847861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fb3b7-6bb3-4b2c-a96b-5d11cfbb2a0b None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:11.848135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fb3b7-6bb3-4b2c-a96b-5d11cfbb2a0b None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:11.848402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fb3b7-6bb3-4b2c-a96b-5d11cfbb2a0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:11.850026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2fb3b7-6bb3-4b2c-a96b-5d11cfbb2a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:11.851436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2fb3b7-6bb3-4b2c-a96b-5d11cfbb2a0b None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:11.852984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 573/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:37:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:11.858741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd472ea-e1ab-400f-8652-0b84b83e497d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:11.954146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd472ea-e1ab-400f-8652-0b84b83e497d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:11.986817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817d0ad-2bed-4078-87b2-19dad2d47630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:11.987354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d0ad-2bed-4078-87b2-19dad2d47630 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:11.987630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d0ad-2bed-4078-87b2-19dad2d47630 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:11.987899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817d0ad-2bed-4078-87b2-19dad2d47630 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:11.988547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d0ad-2bed-4078-87b2-19dad2d47630 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:11.990569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817d0ad-2bed-4078-87b2-19dad2d47630 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:12.010821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 565/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:37:11 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:37:14.164832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf279-6e72-4664-8bce-b925c45d9332 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:14.165378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf279-6e72-4664-8bce-b925c45d9332 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:14.165378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf279-6e72-4664-8bce-b925c45d9332 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:14.166701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 574/1139] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:14.172095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9368d-b9ec-40dc-84b3-c13571e936e2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:14.172500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9368d-b9ec-40dc-84b3-c13571e936e2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:14.172630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9368d-b9ec-40dc-84b3-c13571e936e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:14.190987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f3e9368d-b9ec-40dc-84b3-c13571e936e2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:14.192152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f3e9368d-b9ec-40dc-84b3-c13571e936e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:14.199230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:14.257917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 566/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:14.529983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf279-6e72-4664-8bce-b925c45d9332 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:14.564727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:14.627739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf279-6e72-4664-8bce-b925c45d9332 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:14.661029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb2f19-a2eb-4da9-aa86-fb4450ac224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:14.662210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb2f19-a2eb-4da9-aa86-fb4450ac224a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:14.662479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb2f19-a2eb-4da9-aa86-fb4450ac224a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:14.662740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb2f19-a2eb-4da9-aa86-fb4450ac224a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:14.672287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb2f19-a2eb-4da9-aa86-fb4450ac224a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-d5eb2f19-a2eb-4da9-aa86-fb4450ac224a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:37:14.676934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:37:14.678642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:37:14.679899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 575/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:37:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:37:14.684405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9368d-b9ec-40dc-84b3-c13571e936e2 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:14.778818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9368d-b9ec-40dc-84b3-c13571e936e2 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:14.812232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:14.812762 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:14.813042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:14.813341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:14.813986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:14.814287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:14.815635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:14.823259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 567/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:37:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:14.828891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb2f19-a2eb-4da9-aa86-fb4450ac224a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:14.918852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:14.925080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb2f19-a2eb-4da9-aa86-fb4450ac224a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:14.957303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e5c04-7f55-43f1-9a83-3448dab39114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:14.957923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e5c04-7f55-43f1-9a83-3448dab39114 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:14.958199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e5c04-7f55-43f1-9a83-3448dab39114 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:14.958466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e5c04-7f55-43f1-9a83-3448dab39114 None admin] PATH_INFO: `/v3/users/8e571b9144414b93b577febd5a58f318` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:14.970790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e5c04-7f55-43f1-9a83-3448dab39114 None admin] RBAC: Authorizing `identity:get_user(user_id=8e571b9144414b93b577febd5a58f318)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:14.972206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e5c04-7f55-43f1-9a83-3448dab39114 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:14.973816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 576/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:14 2022] GET /identity/v3/users/8e571b9144414b93b577febd5a58f318 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:14.980213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:14.983925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:15.077168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d52dd-60cb-4c72-8b4f-5b967193fe8d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:15.109588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6f8d1-22b3-4fdb-9f07-82c12e665ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:15.110179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6f8d1-22b3-4fdb-9f07-82c12e665ea0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:15.110459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6f8d1-22b3-4fdb-9f07-82c12e665ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:15.110734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6f8d1-22b3-4fdb-9f07-82c12e665ea0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:15.174393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6f8d1-22b3-4fdb-9f07-82c12e665ea0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:15.175813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e6f8d1-22b3-4fdb-9f07-82c12e665ea0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:15.178814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:15.270365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 568/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:37:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:37:15.276294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e5c04-7f55-43f1-9a83-3448dab39114 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:15.371154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e5c04-7f55-43f1-9a83-3448dab39114 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:15.403551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fa446-4f52-4130-87aa-b5c9a9300de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:15.404161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fa446-4f52-4130-87aa-b5c9a9300de8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:15.404437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fa446-4f52-4130-87aa-b5c9a9300de8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:15.404709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fa446-4f52-4130-87aa-b5c9a9300de8 None admin] PATH_INFO: `/v3/projects/8a48c74c3a5b4fc1bf332ff5f0f847d0` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:15.405843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fa446-4f52-4130-87aa-b5c9a9300de8 None admin] RBAC: Authorizing `identity:get_project(project_id=8a48c74c3a5b4fc1bf332ff5f0f847d0)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:15.407254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020fa446-4f52-4130-87aa-b5c9a9300de8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:15.408829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 577/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:37:15 2022] GET /identity/v3/projects/8a48c74c3a5b4fc1bf332ff5f0f847d0 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:15.414753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6f8d1-22b3-4fdb-9f07-82c12e665ea0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:15.509177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6f8d1-22b3-4fdb-9f07-82c12e665ea0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:15.511391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:15.542951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917b299-660f-40c5-9853-5efbf70ab66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:15.543535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917b299-660f-40c5-9853-5efbf70ab66b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:15.543813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917b299-660f-40c5-9853-5efbf70ab66b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:15.544082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917b299-660f-40c5-9853-5efbf70ab66b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:15.544791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917b299-660f-40c5-9853-5efbf70ab66b None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=8a48c74c3a5b4fc1bf332ff5f0f847d0, user.id=8e571b9144414b93b577febd5a58f318) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:15.545128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917b299-660f-40c5-9853-5efbf70ab66b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:15.546538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917b299-660f-40c5-9853-5efbf70ab66b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:15.551718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 569/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:37:15 2022] GET /identity/v3/role_assignments?user.id=8e571b9144414b93b577febd5a58f318&scope.project.id=8a48c74c3a5b4fc1bf332ff5f0f847d0&role.id=b131b61299154544a627315649090a49 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:17.624714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3064b-c22b-467f-8807-e4d39c3f02c9 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:17.625285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3064b-c22b-467f-8807-e4d39c3f02c9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:17.625285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3064b-c22b-467f-8807-e4d39c3f02c9 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:17.626592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 578/1147] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:17.632352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361019c-d1f0-4068-80b7-2d5d507f38e5 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:17.632726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361019c-d1f0-4068-80b7-2d5d507f38e5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:17.632892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361019c-d1f0-4068-80b7-2d5d507f38e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:17.650462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7361019c-d1f0-4068-80b7-2d5d507f38e5 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:17.651623 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7361019c-d1f0-4068-80b7-2d5d507f38e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:17.715959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 570/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:17.983884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3064b-c22b-467f-8807-e4d39c3f02c9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:18.008195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:18.083042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3064b-c22b-467f-8807-e4d39c3f02c9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:18.115140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586cdb3-fd09-4066-b7d7-13eff7c07a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:18.115751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586cdb3-fd09-4066-b7d7-13eff7c07a40 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:18.116030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586cdb3-fd09-4066-b7d7-13eff7c07a40 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:18.116298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586cdb3-fd09-4066-b7d7-13eff7c07a40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:18.125932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586cdb3-fd09-4066-b7d7-13eff7c07a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-d586cdb3-fd09-4066-b7d7-13eff7c07a40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:37:18.131250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:37:18.132337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:37:18.133604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 579/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:37:18.138178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361019c-d1f0-4068-80b7-2d5d507f38e5 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:18.229109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:18.232790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361019c-d1f0-4068-80b7-2d5d507f38e5 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:18.265608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:18.266199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:18.266477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:18.266747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:18.267395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:18.267695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:18.269045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:18.277955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 571/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:37:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:18.283616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586cdb3-fd09-4066-b7d7-13eff7c07a40 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:18.362139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:18.380708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586cdb3-fd09-4066-b7d7-13eff7c07a40 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:18.412875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b362d2c-fb8c-4f75-9861-8497c5644293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:18.413516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b362d2c-fb8c-4f75-9861-8497c5644293 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:18.413793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b362d2c-fb8c-4f75-9861-8497c5644293 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:18.414064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b362d2c-fb8c-4f75-9861-8497c5644293 None admin] PATH_INFO: `/v3/users/8e571b9144414b93b577febd5a58f318` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:18.415159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b362d2c-fb8c-4f75-9861-8497c5644293 None admin] RBAC: Authorizing `identity:get_user(user_id=8e571b9144414b93b577febd5a58f318)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:18.416571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b362d2c-fb8c-4f75-9861-8497c5644293 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:18.418146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 580/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:18 2022] GET /identity/v3/users/8e571b9144414b93b577febd5a58f318 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:18.423861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:18.520804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac90f5-f0ec-43b3-9153-a3659463525b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:18.552892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e5d0d-b529-47e3-8d9c-5d1b92ed73d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:18.553533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e5d0d-b529-47e3-8d9c-5d1b92ed73d7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:18.553813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e5d0d-b529-47e3-8d9c-5d1b92ed73d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:18.554083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e5d0d-b529-47e3-8d9c-5d1b92ed73d7 None admin] PATH_INFO: `/v3/projects/8a48c74c3a5b4fc1bf332ff5f0f847d0` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:18.555185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e5d0d-b529-47e3-8d9c-5d1b92ed73d7 None admin] RBAC: Authorizing `identity:get_project(project_id=8a48c74c3a5b4fc1bf332ff5f0f847d0)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:18.556589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e5d0d-b529-47e3-8d9c-5d1b92ed73d7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:18.558203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 572/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:37:18 2022] GET /identity/v3/projects/8a48c74c3a5b4fc1bf332ff5f0f847d0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:18.563865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b362d2c-fb8c-4f75-9861-8497c5644293 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:18.658386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b362d2c-fb8c-4f75-9861-8497c5644293 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:18.690942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae548534-b98a-4e80-aa51-0dd28f4a6f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:18.691578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae548534-b98a-4e80-aa51-0dd28f4a6f81 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:18.691854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae548534-b98a-4e80-aa51-0dd28f4a6f81 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:18.692124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae548534-b98a-4e80-aa51-0dd28f4a6f81 None admin] PATH_INFO: `/v3/projects/8a48c74c3a5b4fc1bf332ff5f0f847d0/users/8e571b9144414b93b577febd5a58f318/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:18.694571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae548534-b98a-4e80-aa51-0dd28f4a6f81 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a48c74c3a5b4fc1bf332ff5f0f847d0, user_id=8e571b9144414b93b577febd5a58f318, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:18.695975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae548534-b98a-4e80-aa51-0dd28f4a6f81 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:18.709442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 581/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:37:18 2022] PUT /identity/v3/projects/8a48c74c3a5b4fc1bf332ff5f0f847d0/users/8e571b9144414b93b577febd5a58f318/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:37:20.863650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa9e78-e6fa-48c6-80d6-0a2c70a1d07e None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:20.864284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa9e78-e6fa-48c6-80d6-0a2c70a1d07e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:20.864284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa9e78-e6fa-48c6-80d6-0a2c70a1d07e None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:20.866200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 573/1154] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:20.871465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c4d96-8f0e-49db-874b-0dc1b27c2a11 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:20.871892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c4d96-8f0e-49db-874b-0dc1b27c2a11 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:20.871992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c4d96-8f0e-49db-874b-0dc1b27c2a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:20.890486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ce8c4d96-8f0e-49db-874b-0dc1b27c2a11 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:20.891672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ce8c4d96-8f0e-49db-874b-0dc1b27c2a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:20.894497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:20.957715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 582/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:21.223539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa9e78-e6fa-48c6-80d6-0a2c70a1d07e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:21.253355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:21.323750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa9e78-e6fa-48c6-80d6-0a2c70a1d07e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:21.355045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d26cd8-19d3-47ec-a7c4-16fac22403ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:21.355654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d26cd8-19d3-47ec-a7c4-16fac22403ae None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:21.355928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d26cd8-19d3-47ec-a7c4-16fac22403ae None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:21.356196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d26cd8-19d3-47ec-a7c4-16fac22403ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:21.366832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d26cd8-19d3-47ec-a7c4-16fac22403ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-82d26cd8-19d3-47ec-a7c4-16fac22403ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:37:21.371512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:37:21.372665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:37:21.373795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 574/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:37:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:37:21.378607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c4d96-8f0e-49db-874b-0dc1b27c2a11 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:21.472861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c4d96-8f0e-49db-874b-0dc1b27c2a11 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:21.510047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:21.510572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:21.510852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:21.511119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:21.511770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:21.512070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:21.513449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:21.521248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 583/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:37:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:21.527331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d26cd8-19d3-47ec-a7c4-16fac22403ae None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:21.612250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:21.624846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d26cd8-19d3-47ec-a7c4-16fac22403ae None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:21.656993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f062970-9237-4f45-a5d4-a97b44cc3ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:21.657644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f062970-9237-4f45-a5d4-a97b44cc3ce9 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:21.657917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f062970-9237-4f45-a5d4-a97b44cc3ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:21.658187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f062970-9237-4f45-a5d4-a97b44cc3ce9 None admin] PATH_INFO: `/v3/users/8e571b9144414b93b577febd5a58f318` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:21.659274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f062970-9237-4f45-a5d4-a97b44cc3ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=8e571b9144414b93b577febd5a58f318)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:21.660680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f062970-9237-4f45-a5d4-a97b44cc3ce9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:21.662745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 575/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:21 2022] GET /identity/v3/users/8e571b9144414b93b577febd5a58f318 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:21.668448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:21.671265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:21.765422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc2488-6755-4f98-b811-405a83a43228 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:21.797512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc92668-36ff-48fe-a30f-56545beb63e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:21.798104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc92668-36ff-48fe-a30f-56545beb63e6 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:21.798380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc92668-36ff-48fe-a30f-56545beb63e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:21.798646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc92668-36ff-48fe-a30f-56545beb63e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:21.862410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc92668-36ff-48fe-a30f-56545beb63e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:21.863834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc92668-36ff-48fe-a30f-56545beb63e6 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:21.866121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:21.959311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 584/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:37:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:37:21.965307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f062970-9237-4f45-a5d4-a97b44cc3ce9 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:22.060457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f062970-9237-4f45-a5d4-a97b44cc3ce9 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:22.092461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda12b-0364-48fb-a7ad-1493b0aeec85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:22.093089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda12b-0364-48fb-a7ad-1493b0aeec85 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:22.093369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda12b-0364-48fb-a7ad-1493b0aeec85 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:22.093636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda12b-0364-48fb-a7ad-1493b0aeec85 None admin] PATH_INFO: `/v3/projects/8a48c74c3a5b4fc1bf332ff5f0f847d0` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:22.094720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fda12b-0364-48fb-a7ad-1493b0aeec85 None admin] RBAC: Authorizing `identity:get_project(project_id=8a48c74c3a5b4fc1bf332ff5f0f847d0)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:22.096123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fda12b-0364-48fb-a7ad-1493b0aeec85 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:22.097728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 576/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:37:21 2022] GET /identity/v3/projects/8a48c74c3a5b4fc1bf332ff5f0f847d0 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:22.104293 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc92668-36ff-48fe-a30f-56545beb63e6 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:22.198947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc92668-36ff-48fe-a30f-56545beb63e6 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:22.200415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:22.233037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee227a-7db0-4ad7-95ad-5b590eed10e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:22.233648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee227a-7db0-4ad7-95ad-5b590eed10e7 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:22.233923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee227a-7db0-4ad7-95ad-5b590eed10e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:22.234190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee227a-7db0-4ad7-95ad-5b590eed10e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:22.234897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee227a-7db0-4ad7-95ad-5b590eed10e7 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=8a48c74c3a5b4fc1bf332ff5f0f847d0, user.id=8e571b9144414b93b577febd5a58f318) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:22.235202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee227a-7db0-4ad7-95ad-5b590eed10e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:22.236596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee227a-7db0-4ad7-95ad-5b590eed10e7 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:22.242210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 585/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:37:22 2022] GET /identity/v3/role_assignments?user.id=8e571b9144414b93b577febd5a58f318&scope.project.id=8a48c74c3a5b4fc1bf332ff5f0f847d0&role.id=b131b61299154544a627315649090a49 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:37:24.417880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58002eed-067c-4b1c-b711-c46d8931e7bd None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:24.418423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58002eed-067c-4b1c-b711-c46d8931e7bd None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:24.418423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58002eed-067c-4b1c-b711-c46d8931e7bd None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:24.419779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 577/1162] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:24.424934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72edc8b-c381-406c-a958-f32026e10ebf None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:24.425379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72edc8b-c381-406c-a958-f32026e10ebf None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:24.425498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72edc8b-c381-406c-a958-f32026e10ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:24.445132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b72edc8b-c381-406c-a958-f32026e10ebf None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:24.446296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b72edc8b-c381-406c-a958-f32026e10ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:24.508609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 586/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:24.775380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58002eed-067c-4b1c-b711-c46d8931e7bd None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:24.800073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:24.874828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58002eed-067c-4b1c-b711-c46d8931e7bd None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:24.907471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051813b-d09a-406a-b8cb-0fe3dfa6aa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:24.908406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051813b-d09a-406a-b8cb-0fe3dfa6aa38 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:24.908686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051813b-d09a-406a-b8cb-0fe3dfa6aa38 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:24.908956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051813b-d09a-406a-b8cb-0fe3dfa6aa38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:24.977359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051813b-d09a-406a-b8cb-0fe3dfa6aa38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:24.978794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051813b-d09a-406a-b8cb-0fe3dfa6aa38 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:25.002915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:25.075035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 578/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:37:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:37:25.080914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72edc8b-c381-406c-a958-f32026e10ebf None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:25.169750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:25.175623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72edc8b-c381-406c-a958-f32026e10ebf None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:25.208755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9f1d9-cdb1-4bcc-afbd-74dd279a1c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:25.209395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9f1d9-cdb1-4bcc-afbd-74dd279a1c7a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:25.209668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9f1d9-cdb1-4bcc-afbd-74dd279a1c7a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:25.209930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9f1d9-cdb1-4bcc-afbd-74dd279a1c7a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:25.216257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9f1d9-cdb1-4bcc-afbd-74dd279a1c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:25.217780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9f1d9-cdb1-4bcc-afbd-74dd279a1c7a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-89c9f1d9-cdb1-4bcc-afbd-74dd279a1c7a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:37:25.228710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:37:25.230015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:37:25.230999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 587/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:37:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:37:25.235649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051813b-d09a-406a-b8cb-0fe3dfa6aa38 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:25.330218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051813b-d09a-406a-b8cb-0fe3dfa6aa38 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:25.352447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:25.362775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f34af4-7af4-4fd5-a9b8-df61bcde2ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:25.363358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f34af4-7af4-4fd5-a9b8-df61bcde2ec6 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:25.363629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f34af4-7af4-4fd5-a9b8-df61bcde2ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:25.363893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f34af4-7af4-4fd5-a9b8-df61bcde2ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:25.365077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f34af4-7af4-4fd5-a9b8-df61bcde2ec6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:25.365375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f34af4-7af4-4fd5-a9b8-df61bcde2ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:25.366701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f34af4-7af4-4fd5-a9b8-df61bcde2ec6 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:25.373506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 579/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Fri Sep 9 01:37:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:27.501888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635c1e-b2f9-4e5d-bf2e-eb58b76af34b None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:27.502417 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635c1e-b2f9-4e5d-bf2e-eb58b76af34b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:27.502417 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635c1e-b2f9-4e5d-bf2e-eb58b76af34b None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:27.503736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 588/1167] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:27.509645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42bbbe-f604-4876-bf69-34cd50e25e0d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:27.510024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42bbbe-f604-4876-bf69-34cd50e25e0d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:27.510188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42bbbe-f604-4876-bf69-34cd50e25e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:27.528454 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1c42bbbe-f604-4876-bf69-34cd50e25e0d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:27.529659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1c42bbbe-f604-4876-bf69-34cd50e25e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:27.593452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 580/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:27.862265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6635c1e-b2f9-4e5d-bf2e-eb58b76af34b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:27.960624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6635c1e-b2f9-4e5d-bf2e-eb58b76af34b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:27.992767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f14c97-1a1e-4bde-8937-a01147cd7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:27.993384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f14c97-1a1e-4bde-8937-a01147cd7ed8 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:27.993659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f14c97-1a1e-4bde-8937-a01147cd7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:27.993928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f14c97-1a1e-4bde-8937-a01147cd7ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:27.994580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f14c97-1a1e-4bde-8937-a01147cd7ed8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:27.995928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f14c97-1a1e-4bde-8937-a01147cd7ed8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:28.011991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 589/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Fri Sep 9 01:37:27 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:37:30.170734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057eee5-f8cd-4a4f-a510-eefba985dab5 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:30.171231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057eee5-f8cd-4a4f-a510-eefba985dab5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:30.171231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057eee5-f8cd-4a4f-a510-eefba985dab5 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:30.172556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 581/1170] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:30.178406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f5b4c-b48b-48c7-9c93-3df6cffdb115 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:30.178814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f5b4c-b48b-48c7-9c93-3df6cffdb115 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:30.178952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f5b4c-b48b-48c7-9c93-3df6cffdb115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:30.197249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-bb4f5b4c-b48b-48c7-9c93-3df6cffdb115 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:30.198426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-bb4f5b4c-b48b-48c7-9c93-3df6cffdb115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:30.208609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:30.262876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 590/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:30.530303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057eee5-f8cd-4a4f-a510-eefba985dab5 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:30.627418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057eee5-f8cd-4a4f-a510-eefba985dab5 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:30.644378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:30.661115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d09f8a-8c7e-4aa9-b26a-a157b35f4ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:30.661734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d09f8a-8c7e-4aa9-b26a-a157b35f4ab9 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:30.662012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d09f8a-8c7e-4aa9-b26a-a157b35f4ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:30.662282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d09f8a-8c7e-4aa9-b26a-a157b35f4ab9 None admin] PATH_INFO: `/v3/domains/836c522b3bfc4a528304315b2cd9d494` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:30.672217 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d09f8a-8c7e-4aa9-b26a-a157b35f4ab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=836c522b3bfc4a528304315b2cd9d494)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:30.673670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d09f8a-8c7e-4aa9-b26a-a157b35f4ab9 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:30.675217 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 582/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Fri Sep 9 01:37:30 2022] GET /identity/v3/domains/836c522b3bfc4a528304315b2cd9d494 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:37:30.680963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f5b4c-b48b-48c7-9c93-3df6cffdb115 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:30.775458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f5b4c-b48b-48c7-9c93-3df6cffdb115 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:30.807702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013783a2-9049-4392-8bea-323336bc6723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:30.808230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013783a2-9049-4392-8bea-323336bc6723 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:30.808502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013783a2-9049-4392-8bea-323336bc6723 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:30.808767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013783a2-9049-4392-8bea-323336bc6723 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:30.809435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013783a2-9049-4392-8bea-323336bc6723 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:30.810829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013783a2-9049-4392-8bea-323336bc6723 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:30.830219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:30.837790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 591/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:37:30 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:37:32.999587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d082c4-32ca-4318-a0ef-3fc13bdf1494 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:33.000097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d082c4-32ca-4318-a0ef-3fc13bdf1494 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:33.000205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d082c4-32ca-4318-a0ef-3fc13bdf1494 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:33.001502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 583/1174] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:33.007396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c5ca1-48f3-4f74-b42c-14d88e6b9024 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:33.007812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c5ca1-48f3-4f74-b42c-14d88e6b9024 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:33.007927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c5ca1-48f3-4f74-b42c-14d88e6b9024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:33.026194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-eb4c5ca1-48f3-4f74-b42c-14d88e6b9024 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:33.027351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-eb4c5ca1-48f3-4f74-b42c-14d88e6b9024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:33.092249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 592/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:33.364764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d082c4-32ca-4318-a0ef-3fc13bdf1494 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:33.463684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d082c4-32ca-4318-a0ef-3fc13bdf1494 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:33.494931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3678f79-28c2-41f7-9673-4c8d3a92c275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:33.495545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3678f79-28c2-41f7-9673-4c8d3a92c275 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:33.495822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3678f79-28c2-41f7-9673-4c8d3a92c275 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:33.496094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3678f79-28c2-41f7-9673-4c8d3a92c275 None admin] PATH_INFO: `/v3/domains/836c522b3bfc4a528304315b2cd9d494` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:33.497203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3678f79-28c2-41f7-9673-4c8d3a92c275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=836c522b3bfc4a528304315b2cd9d494)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:33.498615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3678f79-28c2-41f7-9673-4c8d3a92c275 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:33.500147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 584/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Fri Sep 9 01:37:33 2022] GET /identity/v3/domains/836c522b3bfc4a528304315b2cd9d494 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:33.506524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c5ca1-48f3-4f74-b42c-14d88e6b9024 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:33.601702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c5ca1-48f3-4f74-b42c-14d88e6b9024 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:33.626894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:33.634809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdba69-8264-4dda-a36a-dc3259058202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:33.635399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdba69-8264-4dda-a36a-dc3259058202 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:33.635670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdba69-8264-4dda-a36a-dc3259058202 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:33.635933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdba69-8264-4dda-a36a-dc3259058202 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:33.636579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdba69-8264-4dda-a36a-dc3259058202 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:33.638000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdba69-8264-4dda-a36a-dc3259058202 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:33.658592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 593/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:37:33 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:37:35.828084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea276b5-ae89-45ef-9dd5-df9267920161 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:35.828636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea276b5-ae89-45ef-9dd5-df9267920161 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:35.828636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea276b5-ae89-45ef-9dd5-df9267920161 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:35.830005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 585/1178] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:35.835235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0219-b313-4061-afd7-f4baa50bb3f8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:35.835624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0219-b313-4061-afd7-f4baa50bb3f8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:35.835768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da0219-b313-4061-afd7-f4baa50bb3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:35.853840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-80da0219-b313-4061-afd7-f4baa50bb3f8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:35.854998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-80da0219-b313-4061-afd7-f4baa50bb3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:35.919541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 594/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:36.189847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea276b5-ae89-45ef-9dd5-df9267920161 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:36.236781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:36.289873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea276b5-ae89-45ef-9dd5-df9267920161 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:36.321456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f9a3c-d571-4086-85e4-8b4831261fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:36.322061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f9a3c-d571-4086-85e4-8b4831261fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:36.322337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f9a3c-d571-4086-85e4-8b4831261fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:36.322606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f9a3c-d571-4086-85e4-8b4831261fa2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:36.333857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f9a3c-d571-4086-85e4-8b4831261fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-b44f9a3c-d571-4086-85e4-8b4831261fa2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:37:36.338571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:37:36.339694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:37:36.340935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 586/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:37:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:37:36.345651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da0219-b313-4061-afd7-f4baa50bb3f8 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:36.441372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da0219-b313-4061-afd7-f4baa50bb3f8 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:36.455212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:36.478038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:36.479066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:36.479562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:36.480068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:36.481108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:36.481424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:36.482801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:36.492998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 595/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:37:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:36.499324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f9a3c-d571-4086-85e4-8b4831261fa2 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:36.596003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f9a3c-d571-4086-85e4-8b4831261fa2 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:36.602744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:36.628564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4c5d3-c7ce-480e-b17f-21b54919486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:36.629710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4c5d3-c7ce-480e-b17f-21b54919486d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:36.629998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4c5d3-c7ce-480e-b17f-21b54919486d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:36.630521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4c5d3-c7ce-480e-b17f-21b54919486d None admin] PATH_INFO: `/v3/users/5fb181d7671742e59ecf12be40de34f5` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:36.643045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4c5d3-c7ce-480e-b17f-21b54919486d None admin] RBAC: Authorizing `identity:get_user(user_id=5fb181d7671742e59ecf12be40de34f5)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:36.644466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4c5d3-c7ce-480e-b17f-21b54919486d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:36.646693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 587/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:36 2022] GET /identity/v3/users/5fb181d7671742e59ecf12be40de34f5 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:37:36.652545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:36.748254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c6d94-0135-4dbd-ba6d-31bced86eac1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:36.781256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd56f7-b3fb-477b-817c-e9aa3c83cdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:36.781849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd56f7-b3fb-477b-817c-e9aa3c83cdbf None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:36.782128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd56f7-b3fb-477b-817c-e9aa3c83cdbf None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:36.782402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd56f7-b3fb-477b-817c-e9aa3c83cdbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:36.825694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:36.852456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd56f7-b3fb-477b-817c-e9aa3c83cdbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:36.853928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd56f7-b3fb-477b-817c-e9aa3c83cdbf None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:36.949599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 596/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:37:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:37:36.955551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4c5d3-c7ce-480e-b17f-21b54919486d None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:37.049879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4c5d3-c7ce-480e-b17f-21b54919486d None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:37.082671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543407ea-be42-44be-a498-386e1557837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:37.083282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543407ea-be42-44be-a498-386e1557837c None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:37.083559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543407ea-be42-44be-a498-386e1557837c None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:37.083830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543407ea-be42-44be-a498-386e1557837c None admin] PATH_INFO: `/v3/projects/fc53fa2800414c3e8366a3ce03928f98` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:37.084924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543407ea-be42-44be-a498-386e1557837c None admin] RBAC: Authorizing `identity:get_project(project_id=fc53fa2800414c3e8366a3ce03928f98)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:37.086365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543407ea-be42-44be-a498-386e1557837c None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:37.087953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 588/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:37:36 2022] GET /identity/v3/projects/fc53fa2800414c3e8366a3ce03928f98 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:37.094981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd56f7-b3fb-477b-817c-e9aa3c83cdbf None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:37.169866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:37.193239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd56f7-b3fb-477b-817c-e9aa3c83cdbf None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:37.226306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd02c68-16ff-4eae-b7a2-e2de1c798412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:37.226893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02c68-16ff-4eae-b7a2-e2de1c798412 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:37.227170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02c68-16ff-4eae-b7a2-e2de1c798412 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:37.227437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02c68-16ff-4eae-b7a2-e2de1c798412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:37.228145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd02c68-16ff-4eae-b7a2-e2de1c798412 None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=fc53fa2800414c3e8366a3ce03928f98, user.id=5fb181d7671742e59ecf12be40de34f5) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:37.228448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd02c68-16ff-4eae-b7a2-e2de1c798412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:37.229893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd02c68-16ff-4eae-b7a2-e2de1c798412 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:37.235104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 597/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:37:37 2022] GET /identity/v3/role_assignments?user.id=5fb181d7671742e59ecf12be40de34f5&scope.project.id=fc53fa2800414c3e8366a3ce03928f98&role.id=b131b61299154544a627315649090a49 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:39.322569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218539-be4e-4921-9d39-67297a1ec50c None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:39.323094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218539-be4e-4921-9d39-67297a1ec50c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:39.323094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218539-be4e-4921-9d39-67297a1ec50c None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:39.324445 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 589/1186] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:39.329596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b61b1d-26b2-492a-8a08-2fd93a5a432a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:39.329983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b61b1d-26b2-492a-8a08-2fd93a5a432a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:39.330145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b61b1d-26b2-492a-8a08-2fd93a5a432a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:39.347638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c3b61b1d-26b2-492a-8a08-2fd93a5a432a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:39.348830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c3b61b1d-26b2-492a-8a08-2fd93a5a432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:39.411870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 598/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:39.689336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218539-be4e-4921-9d39-67297a1ec50c None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:39.719166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:39.787575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218539-be4e-4921-9d39-67297a1ec50c None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:39.819965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c34017-97e0-4685-a164-700b018d7912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:39.820573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c34017-97e0-4685-a164-700b018d7912 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:39.820850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c34017-97e0-4685-a164-700b018d7912 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:39.821149 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c34017-97e0-4685-a164-700b018d7912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:39.830786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c34017-97e0-4685-a164-700b018d7912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-78c34017-97e0-4685-a164-700b018d7912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:37:39.836081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:37:39.837186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:37:39.838229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 590/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:37:39.843124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b61b1d-26b2-492a-8a08-2fd93a5a432a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:39.909080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:39.943716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b61b1d-26b2-492a-8a08-2fd93a5a432a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:39.975354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:39.975940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:39.976209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:39.976474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:39.977142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:39.977440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:39.978782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:39.986389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 599/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:37:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:39.993302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c34017-97e0-4685-a164-700b018d7912 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:40.081348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:40.089896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c34017-97e0-4685-a164-700b018d7912 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:40.122004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeca6ac-4c64-4e3c-b98b-71972a1e11cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:40.122611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeca6ac-4c64-4e3c-b98b-71972a1e11cc None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:40.122885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeca6ac-4c64-4e3c-b98b-71972a1e11cc None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:40.123155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeca6ac-4c64-4e3c-b98b-71972a1e11cc None admin] PATH_INFO: `/v3/users/5fb181d7671742e59ecf12be40de34f5` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:40.124248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeca6ac-4c64-4e3c-b98b-71972a1e11cc None admin] RBAC: Authorizing `identity:get_user(user_id=5fb181d7671742e59ecf12be40de34f5)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:40.125682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeca6ac-4c64-4e3c-b98b-71972a1e11cc None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:40.127214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 591/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:39 2022] GET /identity/v3/users/5fb181d7671742e59ecf12be40de34f5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:40.133616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:40.227252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8171c8-4537-48ae-a03e-041ab2cc8ebe None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:40.254532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:40.259694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a83b96-2e1a-4447-9233-c5fa8a9159ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:40.260327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83b96-2e1a-4447-9233-c5fa8a9159ff None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:40.260601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83b96-2e1a-4447-9233-c5fa8a9159ff None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:40.260866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83b96-2e1a-4447-9233-c5fa8a9159ff None admin] PATH_INFO: `/v3/projects/fc53fa2800414c3e8366a3ce03928f98` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:40.261983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83b96-2e1a-4447-9233-c5fa8a9159ff None admin] RBAC: Authorizing `identity:get_project(project_id=fc53fa2800414c3e8366a3ce03928f98)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:40.263419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83b96-2e1a-4447-9233-c5fa8a9159ff None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:40.264985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 600/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:37:40 2022] GET /identity/v3/projects/fc53fa2800414c3e8366a3ce03928f98 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:40.270706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeca6ac-4c64-4e3c-b98b-71972a1e11cc None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:40.364603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeca6ac-4c64-4e3c-b98b-71972a1e11cc None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:40.397260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc952370-20c3-400b-81dd-145251aa184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:40.397896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952370-20c3-400b-81dd-145251aa184d None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:40.398176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952370-20c3-400b-81dd-145251aa184d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:40.398450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc952370-20c3-400b-81dd-145251aa184d None admin] PATH_INFO: `/v3/projects/fc53fa2800414c3e8366a3ce03928f98/users/5fb181d7671742e59ecf12be40de34f5/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:40.400413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc952370-20c3-400b-81dd-145251aa184d None admin] RBAC: Authorizing `identity:create_grant(project_id=fc53fa2800414c3e8366a3ce03928f98, user_id=5fb181d7671742e59ecf12be40de34f5, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:40.401856 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc952370-20c3-400b-81dd-145251aa184d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:40.414543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 592/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:37:40 2022] PUT /identity/v3/projects/fc53fa2800414c3e8366a3ce03928f98/users/5fb181d7671742e59ecf12be40de34f5/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:37:42.524553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeceec-f6a7-4a88-a008-b140907de9e6 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:42.525140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeceec-f6a7-4a88-a008-b140907de9e6 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:42.525140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeceec-f6a7-4a88-a008-b140907de9e6 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:42.526437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 601/1193] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:37:42.531786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac55e-9b51-4bbf-bc81-b457205f28dc None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:42.532200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac55e-9b51-4bbf-bc81-b457205f28dc None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:42.532327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac55e-9b51-4bbf-bc81-b457205f28dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:42.549952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9b9ac55e-9b51-4bbf-bc81-b457205f28dc None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:37:42.551127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9b9ac55e-9b51-4bbf-bc81-b457205f28dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:37:42.564632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:42.616128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 593/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:37:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:37:42.881976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeceec-f6a7-4a88-a008-b140907de9e6 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:42.981240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeceec-f6a7-4a88-a008-b140907de9e6 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:43.012794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd32dbb-d880-4641-b3c7-8f04a196cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:43.013435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd32dbb-d880-4641-b3c7-8f04a196cd1f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:43.013713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd32dbb-d880-4641-b3c7-8f04a196cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:43.013981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd32dbb-d880-4641-b3c7-8f04a196cd1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:43.024613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd32dbb-d880-4641-b3c7-8f04a196cd1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-6bd32dbb-d880-4641-b3c7-8f04a196cd1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:37:43.029406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:37:43.030385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:37:43.031595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 602/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:37:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:37:43.037089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ac55e-9b51-4bbf-bc81-b457205f28dc None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:43.129887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ac55e-9b51-4bbf-bc81-b457205f28dc None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:43.163748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:43.164272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:43.164545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:43.164811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:43.165490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:43.165787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:43.167128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:43.174694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 594/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:37:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:43.180412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd32dbb-d880-4641-b3c7-8f04a196cd1f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:43.237597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:43.278380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd32dbb-d880-4641-b3c7-8f04a196cd1f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:43.309570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3994053-3bbb-4b6c-b03d-5d36d62758e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:43.310178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3994053-3bbb-4b6c-b03d-5d36d62758e8 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:43.310454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3994053-3bbb-4b6c-b03d-5d36d62758e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:43.310721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3994053-3bbb-4b6c-b03d-5d36d62758e8 None admin] PATH_INFO: `/v3/users/5fb181d7671742e59ecf12be40de34f5` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:43.311809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3994053-3bbb-4b6c-b03d-5d36d62758e8 None admin] RBAC: Authorizing `identity:get_user(user_id=5fb181d7671742e59ecf12be40de34f5)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:43.313252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3994053-3bbb-4b6c-b03d-5d36d62758e8 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:43.314814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 603/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Fri Sep 9 01:37:43 2022] GET /identity/v3/users/5fb181d7671742e59ecf12be40de34f5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:43.320948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:43.336184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:43.417411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c7878-27ed-4ab4-bbb4-284b605b3431 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:43.449293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2d94e-73fc-4f92-821f-92e66923afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:43.449880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d94e-73fc-4f92-821f-92e66923afa0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:43.450155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d94e-73fc-4f92-821f-92e66923afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:43.450424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2d94e-73fc-4f92-821f-92e66923afa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:43.518714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2d94e-73fc-4f92-821f-92e66923afa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:43.520802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2d94e-73fc-4f92-821f-92e66923afa0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:43.533849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:43.614987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 595/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:37:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:37:43.620887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3994053-3bbb-4b6c-b03d-5d36d62758e8 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:43.722215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3994053-3bbb-4b6c-b03d-5d36d62758e8 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:43.747388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:43.755511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9c128-b7fa-49c2-a1ba-2125ae9675a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:43.756135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c128-b7fa-49c2-a1ba-2125ae9675a4 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:43.756414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c128-b7fa-49c2-a1ba-2125ae9675a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:43.756692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c128-b7fa-49c2-a1ba-2125ae9675a4 None admin] PATH_INFO: `/v3/projects/fc53fa2800414c3e8366a3ce03928f98` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:43.757830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9c128-b7fa-49c2-a1ba-2125ae9675a4 None admin] RBAC: Authorizing `identity:get_project(project_id=fc53fa2800414c3e8366a3ce03928f98)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:43.759244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f9c128-b7fa-49c2-a1ba-2125ae9675a4 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:43.760857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 604/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Fri Sep 9 01:37:43 2022] GET /identity/v3/projects/fc53fa2800414c3e8366a3ce03928f98 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:43.766983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2d94e-73fc-4f92-821f-92e66923afa0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:37:43.864113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2d94e-73fc-4f92-821f-92e66923afa0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:37:43.876581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:43.897616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f5c61-553b-4733-875e-1f9b92e887ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:37:43.898211 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f5c61-553b-4733-875e-1f9b92e887ca None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:37:43.898481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f5c61-553b-4733-875e-1f9b92e887ca None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:37:43.898744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f5c61-553b-4733-875e-1f9b92e887ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:37:43.899450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f5c61-553b-4733-875e-1f9b92e887ca None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=fc53fa2800414c3e8366a3ce03928f98, user.id=5fb181d7671742e59ecf12be40de34f5) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:37:43.899752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f5c61-553b-4733-875e-1f9b92e887ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:37:43.901185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f5c61-553b-4733-875e-1f9b92e887ca None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:37:43.906788 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 596/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:37:43 2022] GET /identity/v3/role_assignments?user.id=5fb181d7671742e59ecf12be40de34f5&scope.project.id=fc53fa2800414c3e8366a3ce03928f98&role.id=b131b61299154544a627315649090a49 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:37:53.751363 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:37:53.904206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=12 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:39.399532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c454f-978e-4a9b-8aa0-c5ddaf32e199 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:39.400149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c454f-978e-4a9b-8aa0-c5ddaf32e199 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:39.400149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c454f-978e-4a9b-8aa0-c5ddaf32e199 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:39.401735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 605/1201] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:38:39.409489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac21140-bbe7-4ba4-8a3e-75b937d59217 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:39.410571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac21140-bbe7-4ba4-8a3e-75b937d59217 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:39.410571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac21140-bbe7-4ba4-8a3e-75b937d59217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:39.442234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-aac21140-bbe7-4ba4-8a3e-75b937d59217 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:38:39.443883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-aac21140-bbe7-4ba4-8a3e-75b937d59217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:38:39.529654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 597/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:38:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 01:38:39.852946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c454f-978e-4a9b-8aa0-c5ddaf32e199 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:39.993425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c454f-978e-4a9b-8aa0-c5ddaf32e199 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:40.031730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:40.032400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:40.032707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:40.033003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:40.045952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:40.047592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:40.063768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:40.064961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:40.065954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:40.065954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:40.065954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:40.065954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 606/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:40.073096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac21140-bbe7-4ba4-8a3e-75b937d59217 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:40.229728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac21140-bbe7-4ba4-8a3e-75b937d59217 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:40.283461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:40.291334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91e8c-ff8e-4f53-8cb1-669674e068ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:40.291957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91e8c-ff8e-4f53-8cb1-669674e068ed None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:40.292233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91e8c-ff8e-4f53-8cb1-669674e068ed None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:40.292503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91e8c-ff8e-4f53-8cb1-669674e068ed None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:40.293200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb91e8c-ff8e-4f53-8cb1-669674e068ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:40.293502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb91e8c-ff8e-4f53-8cb1-669674e068ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:40.294962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb91e8c-ff8e-4f53-8cb1-669674e068ed None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:40.310739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 598/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:40.316583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:40.413027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:40.466351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7d4ba-1c95-4d1e-acd8-5e6c761916da None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:40.512070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e9094-b7f4-422d-82da-001c720f7858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:40.512621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e9094-b7f4-422d-82da-001c720f7858 None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:40.512898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e9094-b7f4-422d-82da-001c720f7858 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:40.513211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e9094-b7f4-422d-82da-001c720f7858 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:40.513901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e9094-b7f4-422d-82da-001c720f7858 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:40.515561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e9094-b7f4-422d-82da-001c720f7858 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:40.548450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 607/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Fri Sep 9 01:38:40 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:38:42.984682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27396d13-e0a8-4016-8ee4-b5baba4e1d41 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:42.985268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27396d13-e0a8-4016-8ee4-b5baba4e1d41 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:42.985268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27396d13-e0a8-4016-8ee4-b5baba4e1d41 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:42.987143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 599/1206] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:38:42.992710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d703ab-905c-4e3a-9246-7b7404ecbeb1 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:42.993154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d703ab-905c-4e3a-9246-7b7404ecbeb1 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:42.993354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d703ab-905c-4e3a-9246-7b7404ecbeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:43.013545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-50d703ab-905c-4e3a-9246-7b7404ecbeb1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:38:43.014826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-50d703ab-905c-4e3a-9246-7b7404ecbeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:38:43.087405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 608/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:38:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:38:43.367089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27396d13-e0a8-4016-8ee4-b5baba4e1d41 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:43.485018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27396d13-e0a8-4016-8ee4-b5baba4e1d41 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:43.526461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd437a9-8d10-497a-a8da-81f64ca08619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:43.527262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd437a9-8d10-497a-a8da-81f64ca08619 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:43.527660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd437a9-8d10-497a-a8da-81f64ca08619 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:43.528000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd437a9-8d10-497a-a8da-81f64ca08619 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:43.544202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd437a9-8d10-497a-a8da-81f64ca08619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-7fd437a9-8d10-497a-a8da-81f64ca08619 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:38:43.550944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:43.552089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:43.552994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 600/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:38:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:43.558968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d703ab-905c-4e3a-9246-7b7404ecbeb1 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:43.605629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:43.675195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d703ab-905c-4e3a-9246-7b7404ecbeb1 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:43.716965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:43.717623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:43.717897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:43.718163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:43.718827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:43.719126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:43.720571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:43.729699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 609/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:38:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:43.736928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd437a9-8d10-497a-a8da-81f64ca08619 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:43.794813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:43.860374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd437a9-8d10-497a-a8da-81f64ca08619 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:43.903942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:43.904602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:43.904878 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:43.905183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:43.920719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:43.922239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:43.939234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:43.940245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:43.941172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:43.941172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:43.941172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:43.941172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 601/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:43.947003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:44.057861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:44.062651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0037c-7b6a-443d-a753-ae1b3c16cc67 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:44.105620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:44.106279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:44.106552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:44.106818 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:44.107535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:44.107839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:44.109308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:44.122129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 610/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:44.128419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:44.275576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1716237-27c3-4704-964c-624b3d9dd16f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:44.278266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:44.316364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:44.317081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:44.317374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:44.317653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:44.328068 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:44.330151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:38:44.346504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:44.348514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:44.350601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 602/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:44.354209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:44.487546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df23f96-6388-42fe-8e12-c329a3248cb0 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:44.525443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:44.526111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:44.526398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:44.526668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:44.527420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:44.527724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:44.529173 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:44.547040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 611/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 01:38:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:38:44.553280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:44.668489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44adbe-3bc2-414b-be51-3dc58e1b6593 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:44.703559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1997783-36d5-45ac-bcd9-2985fcd4491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:44.704271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1997783-36d5-45ac-bcd9-2985fcd4491b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:44.705204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1997783-36d5-45ac-bcd9-2985fcd4491b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:44.705473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1997783-36d5-45ac-bcd9-2985fcd4491b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:44.736237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:44.794854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1997783-36d5-45ac-bcd9-2985fcd4491b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:44.797556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1997783-36d5-45ac-bcd9-2985fcd4491b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:44.941706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 603/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:38:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:38:44.942995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:44.964191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:45.085359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0f29c-3282-409f-8beb-d00ed35bed3a None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:45.129467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:45.130456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:45.130859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:45.131197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:45.145669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:45.147641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:45.165532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:45.166718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:45.167890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:45.167890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:45.167890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:45.167890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 612/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:45.172979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1997783-36d5-45ac-bcd9-2985fcd4491b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:45.238170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:45.282341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1997783-36d5-45ac-bcd9-2985fcd4491b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:45.318653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:45.319256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:45.319536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:45.319807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:45.320444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:45.320745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:45.322386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:45.334929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 604/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:45.341198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:45.427503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:45.465329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06456439-96be-4d4b-82de-0142e46574cd None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:45.507023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:45.507658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:45.507972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:45.508226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:45.515614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:45.517153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:38:45.529519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:45.530870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:45.532325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 613/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:45.537452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:45.654735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82223773-56fa-438c-ad3a-dba55f300aa1 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:45.677626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:45.692474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252445c-5a8f-42a3-87aa-48d6c26e6b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:45.693108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252445c-5a8f-42a3-87aa-48d6c26e6b46 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:45.693471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252445c-5a8f-42a3-87aa-48d6c26e6b46 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:45.693823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252445c-5a8f-42a3-87aa-48d6c26e6b46 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:45.694514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252445c-5a8f-42a3-87aa-48d6c26e6b46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:45.694822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252445c-5a8f-42a3-87aa-48d6c26e6b46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:45.696519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252445c-5a8f-42a3-87aa-48d6c26e6b46 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:45.711419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 605/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:45.717322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:45.830586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f725d42-e17a-4fa3-8345-410978446cd5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:45.862045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:45.868289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4708c9c-6eac-44cc-813a-dd33337c9cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:45.868895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4708c9c-6eac-44cc-813a-dd33337c9cbd None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:45.869196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4708c9c-6eac-44cc-813a-dd33337c9cbd None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:45.869553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4708c9c-6eac-44cc-813a-dd33337c9cbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:45.870306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4708c9c-6eac-44cc-813a-dd33337c9cbd None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=365e1f44e9d849118e64975b4c3d674b) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:45.870614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4708c9c-6eac-44cc-813a-dd33337c9cbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:45.872030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4708c9c-6eac-44cc-813a-dd33337c9cbd None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:45.887853 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 614/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:38:45 2022] GET /identity/v3/role_assignments?user.id=365e1f44e9d849118e64975b4c3d674b&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:48.119387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02436bd3-90ab-423b-887d-2e82bc31608f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:48.119938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02436bd3-90ab-423b-887d-2e82bc31608f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:48.119938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02436bd3-90ab-423b-887d-2e82bc31608f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:48.121312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 606/1220] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:38:48.127406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcaaf5-5d2e-4605-9ab2-cd3f890311a4 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:48.127852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcaaf5-5d2e-4605-9ab2-cd3f890311a4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:48.128021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcaaf5-5d2e-4605-9ab2-cd3f890311a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:48.151262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b9fcaaf5-5d2e-4605-9ab2-cd3f890311a4 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:38:48.152764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b9fcaaf5-5d2e-4605-9ab2-cd3f890311a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:38:48.231315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 615/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:38:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:38:48.503882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02436bd3-90ab-423b-887d-2e82bc31608f None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:48.614275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02436bd3-90ab-423b-887d-2e82bc31608f None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:48.650713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd1554-3218-4d5b-a4c4-509bfd03a654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:48.651397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd1554-3218-4d5b-a4c4-509bfd03a654 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:48.651716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd1554-3218-4d5b-a4c4-509bfd03a654 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:48.651992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd1554-3218-4d5b-a4c4-509bfd03a654 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:48.663119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd1554-3218-4d5b-a4c4-509bfd03a654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-61fd1554-3218-4d5b-a4c4-509bfd03a654 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:38:48.667929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:48.669020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:48.669976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 607/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:48.676446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcaaf5-5d2e-4605-9ab2-cd3f890311a4 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:48.781221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcaaf5-5d2e-4605-9ab2-cd3f890311a4 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:48.807995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:48.819779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:48.820387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:48.820660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:48.820929 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:48.821655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] RBAC: Adding query filter params (name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:48.821953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:48.823316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:48.831792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 616/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:38:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:48.838116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd1554-3218-4d5b-a4c4-509bfd03a654 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:48.882939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:48.947007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd1554-3218-4d5b-a4c4-509bfd03a654 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:48.981979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:48.982572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:48.982854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:48.983122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:48.993649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:48.995075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:49.009913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:49.011251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:49.012404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:49.012404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:49.012404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:49.012404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 608/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:49.017472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:49.120910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12910bf7-556c-41c5-b535-e059f9373375 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:49.155432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:49.156052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:49.156288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:49.156550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:49.157208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:49.157559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:49.158913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:49.171034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 617/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:49.177000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:49.282933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:49.290563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e96ec-56ee-4719-9b83-763d7bbd1d65 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:49.328760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:49.329512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:49.329791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:49.330064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:49.338427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:49.339849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:38:49.351243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:49.352350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:49.353512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 609/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:49.358304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:49.406626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:49.462249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37cd74-7ce7-469f-8f69-525ef36a9a03 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:49.499950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:49.500558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:49.500832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:49.501164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:49.501909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:49.502214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:49.503629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:49.521592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 618/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 01:38:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:49.527611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:49.639413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bade7-66d8-407b-ab11-0d06ef896120 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:49.680471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:49.681077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:49.681407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:49.681674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:49.694381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:49.695888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:49.711618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:49.712747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:49.713949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:49.713949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:49.713949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:49.717822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 610/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:49.718855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:49.830538 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230af4f9-4e72-4e38-86b6-9a367157c33d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:49.832873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:49.869814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:49.870653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:49.871096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:49.871540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:49.872275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:49.872735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:49.874304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:49.887035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 619/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:49.893880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:49.903803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:50.002192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f349c8-cbac-4913-afb5-8250f2bfe10b None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:50.036297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:50.036970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:50.037396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:50.037715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:50.045382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:50.046819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:38:50.060768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:50.062027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:50.063134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 611/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:50.069536 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:50.177865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395216a-83b5-4d29-b0bd-f235d236bc03 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:50.212114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80391620-b18b-4953-b363-ce66de5a8d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:50.212714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80391620-b18b-4953-b363-ce66de5a8d45 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:50.213012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80391620-b18b-4953-b363-ce66de5a8d45 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:50.213341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80391620-b18b-4953-b363-ce66de5a8d45 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:50.214020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80391620-b18b-4953-b363-ce66de5a8d45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:50.214375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80391620-b18b-4953-b363-ce66de5a8d45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:50.216081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80391620-b18b-4953-b363-ce66de5a8d45 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:50.229502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 620/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:50.235276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:50.313206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:50.350261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edf6a3-1d96-4497-ac54-ffc3d4a225e4 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:50.384140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c5be2-8dba-49b2-8732-040531ca30d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:50.384788 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c5be2-8dba-49b2-8732-040531ca30d7 None admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:50.385078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c5be2-8dba-49b2-8732-040531ca30d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:50.385360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c5be2-8dba-49b2-8732-040531ca30d7 None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/365e1f44e9d849118e64975b4c3d674b/roles/a48f239bb178461d9d0ab9daaef7e154` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:50.401273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c5be2-8dba-49b2-8732-040531ca30d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=365e1f44e9d849118e64975b4c3d674b, role_id=a48f239bb178461d9d0ab9daaef7e154)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:50.402751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c5be2-8dba-49b2-8732-040531ca30d7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:50.417704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 612/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:38:50 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/365e1f44e9d849118e64975b4c3d674b/roles/a48f239bb178461d9d0ab9daaef7e154 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:38:52.564387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1dd8a-d208-491a-b9e2-9eaafe25d641 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:52.564961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1dd8a-d208-491a-b9e2-9eaafe25d641 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:52.564961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1dd8a-d208-491a-b9e2-9eaafe25d641 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:52.566708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 621/1233] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:38:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:38:52.572094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d06b8-ab90-4a5b-b3af-b912adf2602e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:52.572614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d06b8-ab90-4a5b-b3af-b912adf2602e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:52.572614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d06b8-ab90-4a5b-b3af-b912adf2602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:52.590888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ef4d06b8-ab90-4a5b-b3af-b912adf2602e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:38:52.592065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ef4d06b8-ab90-4a5b-b3af-b912adf2602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:38:52.661010 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 613/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:38:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:38:52.937930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1dd8a-d208-491a-b9e2-9eaafe25d641 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:52.964657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:53.059567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1dd8a-d208-491a-b9e2-9eaafe25d641 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:53.102521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0c6f2-614f-4d46-a0c8-d5d6d3d1547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:53.103153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c6f2-614f-4d46-a0c8-d5d6d3d1547f None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:53.103431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c6f2-614f-4d46-a0c8-d5d6d3d1547f None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:53.103696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c6f2-614f-4d46-a0c8-d5d6d3d1547f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:53.115977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0c6f2-614f-4d46-a0c8-d5d6d3d1547f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-ffc0c6f2-614f-4d46-a0c8-d5d6d3d1547f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:38:53.120775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:53.121748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:53.122605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 622/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Fri Sep 9 01:38:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:53.128248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d06b8-ab90-4a5b-b3af-b912adf2602e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:53.179089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:53.241950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d06b8-ab90-4a5b-b3af-b912adf2602e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:53.287779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:53.289118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:53.289737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:53.290067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:53.290848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] RBAC: Adding query filter params (name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:53.291206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:53.292971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:53.307923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 614/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Fri Sep 9 01:38:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:38:53.310442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0c6f2-614f-4d46-a0c8-d5d6d3d1547f None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:53.412494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:53.430405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0c6f2-614f-4d46-a0c8-d5d6d3d1547f None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:53.468990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:53.470219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:53.470503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:53.470773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:53.482187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:53.483609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:53.500898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:53.507389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:53.508352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:53.508352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:53.508352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:53.508352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 623/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:53.508700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:53.619709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c5077-ae3a-42ea-a75f-2c9a24c40313 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:53.630174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:53.661530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:53.662471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:53.662750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:53.663078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:53.663840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:53.664252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:53.665903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:53.679886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 615/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:53.685960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:53.798810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11b05-c84c-410c-8c7b-624b4194ea64 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:53.850614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:53.851615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:53.852062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:53.852506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:53.855838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:53.864802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:53.866578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:38:53.882653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:53.883891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:53.885034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 624/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:53.890300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:54.005677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109d963-74b4-4d69-9382-0cba136a0800 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:54.041265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:54.041866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:54.042139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:54.042406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:54.043120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:54.043419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:54.044851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:54.062258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 616/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 01:38:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:54.068601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:54.177142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58517676-755e-4bd1-9eee-26fd7c9f2f72 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:54.223027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8857e8b-8b24-441f-90be-2b5106429867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:54.223726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8857e8b-8b24-441f-90be-2b5106429867 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:54.224078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8857e8b-8b24-441f-90be-2b5106429867 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:54.224355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8857e8b-8b24-441f-90be-2b5106429867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:54.292698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:54.301309 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8857e8b-8b24-441f-90be-2b5106429867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:54.302818 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8857e8b-8b24-441f-90be-2b5106429867 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:54.423596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 625/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:38:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:38:54.425011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:54.458175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:54.548497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b142e2-41ce-4216-bc47-8d87d96bccd0 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:54.587319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:54.587959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:54.588239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:54.588506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:54.602989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:54.604603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:54.620642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:54.627405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:54.628301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:54.628301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:54.628301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:54.628301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 617/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:54.628536 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8857e8b-8b24-441f-90be-2b5106429867 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:54.730728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:54.738714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8857e8b-8b24-441f-90be-2b5106429867 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:54.772633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:54.773347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:54.773629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:54.773904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:54.774563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:54.774869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:54.776347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:54.790329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 626/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:54.795574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:54.885622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:54.914005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4594515-88b4-475f-ad48-19f33c08f794 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:54.951850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:54.952476 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:54.952754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:54.953021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:54.961054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:54.962607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:38:54.975325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:54.976377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:54.977386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 618/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:54.984248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:55.095332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa329b5-7396-4bda-98f7-abb506c8accc None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:55.132762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a20dd-ec97-4b79-9796-3bbfd1a70029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:55.133413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a20dd-ec97-4b79-9796-3bbfd1a70029 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:55.133690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a20dd-ec97-4b79-9796-3bbfd1a70029 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:55.134003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a20dd-ec97-4b79-9796-3bbfd1a70029 None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:55.134796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a20dd-ec97-4b79-9796-3bbfd1a70029 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:55.135141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a20dd-ec97-4b79-9796-3bbfd1a70029 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:55.136795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a20dd-ec97-4b79-9796-3bbfd1a70029 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:55.149260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 627/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:55.155932 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:55.269351 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab78ab5-94ae-48d8-872b-6d13f9cbbb3a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:55.321952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bb534-c80a-408b-9246-72d83c318c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:55.322904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb534-c80a-408b-9246-72d83c318c29 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:55.323314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb534-c80a-408b-9246-72d83c318c29 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:55.323766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb534-c80a-408b-9246-72d83c318c29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:55.325301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bb534-c80a-408b-9246-72d83c318c29 None admin] RBAC: Adding query filter params (role.id=a48f239bb178461d9d0ab9daaef7e154, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=365e1f44e9d849118e64975b4c3d674b) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:55.325870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bb534-c80a-408b-9246-72d83c318c29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:55.328132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bb534-c80a-408b-9246-72d83c318c29 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:55.336236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 619/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:38:55 2022] GET /identity/v3/role_assignments?user.id=365e1f44e9d849118e64975b4c3d674b&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=a48f239bb178461d9d0ab9daaef7e154 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:38:57.713184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e67a8-af54-40c1-bc10-256b9870c9cb None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:57.713798 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e67a8-af54-40c1-bc10-256b9870c9cb None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:57.713798 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e67a8-af54-40c1-bc10-256b9870c9cb None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:57.715331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 628/1247] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:38:57.721588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954715ba-3835-496d-9476-e4e0de2b537e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:57.722111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954715ba-3835-496d-9476-e4e0de2b537e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:57.722323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954715ba-3835-496d-9476-e4e0de2b537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:57.730005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:57.744750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-954715ba-3835-496d-9476-e4e0de2b537e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:38:57.746294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-954715ba-3835-496d-9476-e4e0de2b537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:38:57.822692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 620/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:38:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:38:58.114530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e67a8-af54-40c1-bc10-256b9870c9cb None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:58.134451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:58.224025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e67a8-af54-40c1-bc10-256b9870c9cb None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:58.261677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66217a-000d-417b-9953-9a41cb9fd5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:58.262394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66217a-000d-417b-9953-9a41cb9fd5b1 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:58.262612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66217a-000d-417b-9953-9a41cb9fd5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:58.262921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66217a-000d-417b-9953-9a41cb9fd5b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:58.275234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66217a-000d-417b-9953-9a41cb9fd5b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-5f66217a-000d-417b-9953-9a41cb9fd5b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:38:58.280688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:58.281697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:58.283068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 629/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:38:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:58.288881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954715ba-3835-496d-9476-e4e0de2b537e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:58.397646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954715ba-3835-496d-9476-e4e0de2b537e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:58.430112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:58.436002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:58.436610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:58.436885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:58.437253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:58.437948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:58.438336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:58.439801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:58.447910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 621/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:38:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:58.453666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66217a-000d-417b-9953-9a41cb9fd5b1 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:58.540428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:58.565410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66217a-000d-417b-9953-9a41cb9fd5b1 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:58.609437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02869-f850-4699-babe-b341755b702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:58.610616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02869-f850-4699-babe-b341755b702d None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:58.610899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02869-f850-4699-babe-b341755b702d None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:58.611162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02869-f850-4699-babe-b341755b702d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:58.622506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d02869-f850-4699-babe-b341755b702d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:58.623970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d02869-f850-4699-babe-b341755b702d None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-81d02869-f850-4699-babe-b341755b702d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:58.640584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:58.642024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:58.643324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:58.643324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:58.643324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:58.643324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 630/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:58.647975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:58.752924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4bd38-f4b9-4731-b7de-52b938c618ca None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:58.788983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:58.789615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:58.789892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:58.790161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:58.790800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:58.791099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:58.792459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:58.809318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 622/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:58.810514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02869-f850-4699-babe-b341755b702d None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:58.917525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02869-f850-4699-babe-b341755b702d None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:58.944573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:58.954360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:58.955018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:58.955295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:58.955561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:58.963753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:58.965247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:38:58.977188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:58.979064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:38:58.980080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 631/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:58.984164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:59.040419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:59.086281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c331aa-c381-4396-af55-ce955f6bf81e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:59.120674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00668299-3f47-40b5-8172-1c231463939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:59.121327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00668299-3f47-40b5-8172-1c231463939f None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:59.121602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00668299-3f47-40b5-8172-1c231463939f None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:59.121867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00668299-3f47-40b5-8172-1c231463939f None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:59.122582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00668299-3f47-40b5-8172-1c231463939f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:59.122899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00668299-3f47-40b5-8172-1c231463939f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:59.124285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00668299-3f47-40b5-8172-1c231463939f None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:59.142235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 623/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 01:38:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:59.147970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:59.247401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3db8db-fb26-489a-bf8a-daa2819c3068 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:59.285724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad708f2-6673-4c74-87e7-985fb77304b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:59.286424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad708f2-6673-4c74-87e7-985fb77304b4 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:59.286735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad708f2-6673-4c74-87e7-985fb77304b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:59.287054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad708f2-6673-4c74-87e7-985fb77304b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:59.341479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:59.354162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad708f2-6673-4c74-87e7-985fb77304b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:59.355578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad708f2-6673-4c74-87e7-985fb77304b4 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:59.448297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 632/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:38:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:38:59.454644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00668299-3f47-40b5-8172-1c231463939f None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:59.550129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00668299-3f47-40b5-8172-1c231463939f None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:59.562589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:59.583881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:59.584497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:59.584773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:59.585044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:59.594791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:59.596211 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:38:59.610887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:59.611965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:38:59.613241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:38:59.613241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:38:59.613241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:59.613241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 624/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:59.617562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad708f2-6673-4c74-87e7-985fb77304b4 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:59.706679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:38:59.718926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad708f2-6673-4c74-87e7-985fb77304b4 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:59.755502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:59.756097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:59.756373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:59.756640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:59.757298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:38:59.757607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:59.758958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:59.770390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 633/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:38:59.776195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:38:59.873872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf371ad7-2037-401e-8c22-875508f7a357 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:38:59.906535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:38:59.907678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:38:59.907946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:38:59.908210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:38:59.914506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:38:59.915935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:38:59.927642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:38:59.928683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:38:59.929990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 625/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:38:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:38:59.934502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:00.033886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e2924-40d9-4772-baaa-9d640762a093 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:00.066191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875bf3d-f2c4-49db-a42c-f709fc41693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:00.066783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875bf3d-f2c4-49db-a42c-f709fc41693a None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:00.067061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875bf3d-f2c4-49db-a42c-f709fc41693a None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:00.067329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875bf3d-f2c4-49db-a42c-f709fc41693a None admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:00.068006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875bf3d-f2c4-49db-a42c-f709fc41693a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:00.068316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875bf3d-f2c4-49db-a42c-f709fc41693a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:00.069756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875bf3d-f2c4-49db-a42c-f709fc41693a None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:00.081695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 634/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:38:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:00.087819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:00.112944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:00.185351 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3a69-600c-4c47-b256-1b4a1aab0a71 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:00.218755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b704ea4-676f-4e52-b5d2-5867b7dfbc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:00.219932 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b704ea4-676f-4e52-b5d2-5867b7dfbc7d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:00.220211 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b704ea4-676f-4e52-b5d2-5867b7dfbc7d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:00.220480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b704ea4-676f-4e52-b5d2-5867b7dfbc7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:00.221250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b704ea4-676f-4e52-b5d2-5867b7dfbc7d None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=365e1f44e9d849118e64975b4c3d674b) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:00.221538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b704ea4-676f-4e52-b5d2-5867b7dfbc7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:00.222941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b704ea4-676f-4e52-b5d2-5867b7dfbc7d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:00.228126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 626/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:39:00 2022] GET /identity/v3/role_assignments?user.id=365e1f44e9d849118e64975b4c3d674b&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=b131b61299154544a627315649090a49 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:02.393274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eb8d9-f91c-4195-adf6-80cbe8ff35c3 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:02.393796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eb8d9-f91c-4195-adf6-80cbe8ff35c3 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:02.393796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eb8d9-f91c-4195-adf6-80cbe8ff35c3 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:02.395151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 635/1261] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:39:02.400296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda59a-2bc3-41cb-be47-65079a0ee105 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:02.400675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda59a-2bc3-41cb-be47-65079a0ee105 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:02.400843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda59a-2bc3-41cb-be47-65079a0ee105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:02.418572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-18fda59a-2bc3-41cb-be47-65079a0ee105 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:39:02.419740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-18fda59a-2bc3-41cb-be47-65079a0ee105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:39:02.487412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 627/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:39:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:39:02.753456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eb8d9-f91c-4195-adf6-80cbe8ff35c3 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:02.764308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:02.850867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eb8d9-f91c-4195-adf6-80cbe8ff35c3 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:02.883646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e6e5-65d7-4b78-b179-072369ee4fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:02.884258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e6e5-65d7-4b78-b179-072369ee4fc5 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:02.884534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e6e5-65d7-4b78-b179-072369ee4fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:02.884802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e6e5-65d7-4b78-b179-072369ee4fc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:02.895108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2e6e5-65d7-4b78-b179-072369ee4fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e3e2e6e5-65d7-4b78-b179-072369ee4fc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:39:02.899892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:02.900938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:39:02.902066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 636/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:39:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:02.907838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda59a-2bc3-41cb-be47-65079a0ee105 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:02.924956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:03.001633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda59a-2bc3-41cb-be47-65079a0ee105 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:03.034169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:03.034703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:03.034991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:03.035237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:03.035881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:03.036177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:03.037553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:03.045225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 628/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:39:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:03.051425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e6e5-65d7-4b78-b179-072369ee4fc5 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:03.117055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:03.149213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e6e5-65d7-4b78-b179-072369ee4fc5 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:03.181516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:03.182119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:03.182395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:03.182662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:03.192809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:03.194266 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:39:03.208945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:03.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:39:03.210999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:39:03.210999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:39:03.210999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:39:03.210999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 637/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:03.215809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:03.298172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:03.317624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadd1b5-5e88-4dc7-8aba-29b7e0d66798 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:03.351418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:03.352011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:03.352286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:03.352556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:03.353291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:03.353593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:03.354947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:03.366913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 629/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:03.372647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:03.467214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d1938-98b6-4237-8f75-2e7e6c7aeda9 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:03.486455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:03.499963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:03.500593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:03.500865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:03.501172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:03.508380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:03.509842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:39:03.520447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:03.521586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:39:03.522551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 638/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:39:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:03.527163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:03.622114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7434c2-cab7-4c2b-ae25-0e3f12d365f7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:03.656113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:03.656701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:03.656976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:03.657269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:03.657974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:03.658275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:03.659682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:03.667938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:03.676296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 630/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 01:39:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:03.682075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:03.778077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045f590-38d9-41e1-a6ef-7b3021fba8d6 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:03.810169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:03.810774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:03.811046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:03.811310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:03.821320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:03.822723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:39:03.836573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:03.837696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:39:03.838568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:39:03.838568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:39:03.838568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:39:03.838568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 639/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:03.843360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:03.941369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1784802-cd38-4acb-b792-f67a285ee3da None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:03.974388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:03.974994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:03.975250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:03.975519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:03.976142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:03.976439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:03.977806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:03.988549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 631/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:03.994292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:04.025291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:04.089350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37112b2-fe44-4980-9b06-78873aeb1133 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:04.120699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:04.121326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:04.121599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:04.121860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:04.128135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:04.129962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:39:04.140772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:04.142299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:39:04.143304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 640/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:04.148186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:04.204266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:04.245398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b4438-03ea-4cbe-af9f-908afed42d60 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:04.285183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc838c59-1bb3-45b7-9dda-74978982395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:04.285786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838c59-1bb3-45b7-9dda-74978982395d None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:04.286068 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838c59-1bb3-45b7-9dda-74978982395d None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:04.286340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838c59-1bb3-45b7-9dda-74978982395d None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:04.287022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc838c59-1bb3-45b7-9dda-74978982395d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:04.287325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc838c59-1bb3-45b7-9dda-74978982395d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:04.288753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc838c59-1bb3-45b7-9dda-74978982395d None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:04.300359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 632/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:04.306350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:04.393547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:04.402255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29966068-1d7e-480c-831c-4b03a9faa827 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:04.434082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf554edf-3067-4eec-91fb-1ebe80d366d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:04.434711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554edf-3067-4eec-91fb-1ebe80d366d3 None admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:04.434983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554edf-3067-4eec-91fb-1ebe80d366d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:04.435250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554edf-3067-4eec-91fb-1ebe80d366d3 None admin] PATH_INFO: `/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/365e1f44e9d849118e64975b4c3d674b/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:04.437107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf554edf-3067-4eec-91fb-1ebe80d366d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=819c1fdfa2a8401f960f551c31cb82e6, user_id=365e1f44e9d849118e64975b4c3d674b, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:04.439047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf554edf-3067-4eec-91fb-1ebe80d366d3 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:04.450957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 641/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Fri Sep 9 01:39:04 2022] PUT /identity/v3/projects/819c1fdfa2a8401f960f551c31cb82e6/users/365e1f44e9d849118e64975b4c3d674b/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:39:06.562107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6a56b-a8b8-4caf-a1a6-a50e72944bf1 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:06.562643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6a56b-a8b8-4caf-a1a6-a50e72944bf1 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:06.562643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6a56b-a8b8-4caf-a1a6-a50e72944bf1 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:06.564504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 633/1274] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:39:06.569568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e81029-7d02-40d5-b7c8-06a16c079f03 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:06.569981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e81029-7d02-40d5-b7c8-06a16c079f03 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:06.570105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e81029-7d02-40d5-b7c8-06a16c079f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:06.589038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-99e81029-7d02-40d5-b7c8-06a16c079f03 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:39:06.590241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-99e81029-7d02-40d5-b7c8-06a16c079f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:39:06.654423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 642/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:39:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:39:06.922086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6a56b-a8b8-4caf-a1a6-a50e72944bf1 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:07.023800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6a56b-a8b8-4caf-a1a6-a50e72944bf1 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:07.041037 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:07.057518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25be2-cd03-4a7d-8eee-e94f360b0c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:07.058128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25be2-cd03-4a7d-8eee-e94f360b0c5e None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:07.058401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25be2-cd03-4a7d-8eee-e94f360b0c5e None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:07.058665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25be2-cd03-4a7d-8eee-e94f360b0c5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:07.068140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd25be2-cd03-4a7d-8eee-e94f360b0c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-bbd25be2-cd03-4a7d-8eee-e94f360b0c5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 01:39:07.072852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:07.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:07.074985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 634/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Fri Sep 9 01:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:07.081943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e81029-7d02-40d5-b7c8-06a16c079f03 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:07.158850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:07.176335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e81029-7d02-40d5-b7c8-06a16c079f03 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:07.209479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:07.210072 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:07.210347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:07.210616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:07.211260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:07.211559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:07.212915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:07.220638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 643/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Fri Sep 9 01:39:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:07.226350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25be2-cd03-4a7d-8eee-e94f360b0c5e None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:07.330246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25be2-cd03-4a7d-8eee-e94f360b0c5e None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:07.362580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:07.363199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:07.363476 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:07.363747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:07.373387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:07.374808 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:39:07.390358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:07.392172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:39:07.393197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:39:07.393197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:39:07.393197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:07.393197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 635/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:07.397762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:07.494339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8f254-e24d-40d0-a48f-ab065b831377 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:07.526497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:07.527081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:07.527355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:07.527621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:07.528242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:07.528541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:07.529916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:07.541562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 644/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:07.546942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:07.576843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:07.644245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa144-db28-4e8a-a7f4-fba67b35c9ba None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:07.676154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:07.676757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:07.677027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:07.677321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:07.684244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:07.685692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 01:39:07.696283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:07.697397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:07.698421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 636/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:39:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:07.703876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:07.707618 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:07.800246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9bf3c-1080-4207-8f2e-004501934861 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:07.833448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:07.834035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:07.834311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:07.834581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:07.835282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:07.835582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:07.836980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:07.853730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 645/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Fri Sep 9 01:39:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:07.859317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:07.947658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:07.957722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a005c-96bd-4b98-9fdb-cb637a0fa7b8 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:07.989994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a40d90-5690-4f13-8829-cb01271d8253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:07.990579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a40d90-5690-4f13-8829-cb01271d8253 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:07.990855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a40d90-5690-4f13-8829-cb01271d8253 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:07.991119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a40d90-5690-4f13-8829-cb01271d8253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:08.055483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a40d90-5690-4f13-8829-cb01271d8253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:08.056848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a40d90-5690-4f13-8829-cb01271d8253 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:08.145247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:08.155628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 637/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Fri Sep 9 01:39:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 01:39:08.162109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:08.221069 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:08.264246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47500ea8-2f6a-4bea-bec5-ca81e205ae09 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:08.298010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:08.298632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:08.298911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:08.299179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:08.308843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:08.310243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 01:39:08.325206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:08.326357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 01:39:08.327638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 01:39:08.327638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 01:39:08.327638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:39:08.327638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 646/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:08.332554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a40d90-5690-4f13-8829-cb01271d8253 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:08.428120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a40d90-5690-4f13-8829-cb01271d8253 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:08.463376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:08.463969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:08.464246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:08.464519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:08.465177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:08.465482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:08.466833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:08.480420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 638/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Fri Sep 9 01:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:08.486015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:08.582158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f287e-35e3-4057-a694-27b712423b20 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:08.594873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:08.614475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:08.615068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:08.615343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:08.615613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] PATH_INFO: `/v3/projects/service` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:08.621926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:08.623333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 01:39:08.634466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:08.635479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: ERROR keystone.server.flask.application Sep 09 01:39:08.636586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 647/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Fri Sep 9 01:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:08.641758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:08.690089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:08.738614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e5b48-785d-4814-bc47-cc2a8398badd None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:08.771263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9f855-093b-47f0-8627-d292ec80ce17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:08.771849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9f855-093b-47f0-8627-d292ec80ce17 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:08.772124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9f855-093b-47f0-8627-d292ec80ce17 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:08.772391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9f855-093b-47f0-8627-d292ec80ce17 None admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:08.773081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9f855-093b-47f0-8627-d292ec80ce17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:08.773437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9f855-093b-47f0-8627-d292ec80ce17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:08.774834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9f855-093b-47f0-8627-d292ec80ce17 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:08.786741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 639/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Fri Sep 9 01:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:08.792795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:08.889304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56a711-7cfd-4f38-bfd0-0251dfb619df None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:08.921643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391156f6-151a-47df-aa9f-736ac502f3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:08.922238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391156f6-151a-47df-aa9f-736ac502f3bf None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:08.922511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391156f6-151a-47df-aa9f-736ac502f3bf None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:08.922778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391156f6-151a-47df-aa9f-736ac502f3bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:08.923490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391156f6-151a-47df-aa9f-736ac502f3bf None admin] RBAC: Adding query filter params (role.id=b131b61299154544a627315649090a49, scope.project.id=819c1fdfa2a8401f960f551c31cb82e6, user.id=365e1f44e9d849118e64975b4c3d674b) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:08.923793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391156f6-151a-47df-aa9f-736ac502f3bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:08.925232 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391156f6-151a-47df-aa9f-736ac502f3bf None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:08.930756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 648/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Fri Sep 9 01:39:08 2022] GET /identity/v3/role_assignments?user.id=365e1f44e9d849118e64975b4c3d674b&scope.project.id=819c1fdfa2a8401f960f551c31cb82e6&role.id=b131b61299154544a627315649090a49 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:11.103085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275861e0-22f7-48f2-8f00-6780cfeb530b None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:11.103629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275861e0-22f7-48f2-8f00-6780cfeb530b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:11.103629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275861e0-22f7-48f2-8f00-6780cfeb530b None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:11.104943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 640/1288] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:39:11.110662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db8b9-0667-4101-9685-a8cb515d4f17 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:11.111103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db8b9-0667-4101-9685-a8cb515d4f17 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:11.111208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db8b9-0667-4101-9685-a8cb515d4f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:11.128829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-432db8b9-0667-4101-9685-a8cb515d4f17 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:39:11.130285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-432db8b9-0667-4101-9685-a8cb515d4f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:39:11.158233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:11.195577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 649/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:39:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:39:11.458112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275861e0-22f7-48f2-8f00-6780cfeb530b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:11.555996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275861e0-22f7-48f2-8f00-6780cfeb530b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:11.566108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:11.588647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:11.589281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:11.589554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:11.589818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:11.590386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:11.591720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:11.600758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:11.601984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:11.603029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 641/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:39:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:11.607677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432db8b9-0667-4101-9685-a8cb515d4f17 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:11.702721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432db8b9-0667-4101-9685-a8cb515d4f17 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:11.735474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69499d4f-8c65-4d3b-9736-3c9f4eb01476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:11.735998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69499d4f-8c65-4d3b-9736-3c9f4eb01476 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:11.736270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69499d4f-8c65-4d3b-9736-3c9f4eb01476 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:11.736533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69499d4f-8c65-4d3b-9736-3c9f4eb01476 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:11.737207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69499d4f-8c65-4d3b-9736-3c9f4eb01476 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:11.737502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69499d4f-8c65-4d3b-9736-3c9f4eb01476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:11.738835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69499d4f-8c65-4d3b-9736-3c9f4eb01476 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:11.743412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 650/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 9 01:39:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:11.748888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:11.845882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076db41-c5eb-4ce5-a757-e127ad822cc7 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:11.877964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b019b4-8a45-4af6-8c8a-43053ce57e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:11.878547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b019b4-8a45-4af6-8c8a-43053ce57e80 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:11.878822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b019b4-8a45-4af6-8c8a-43053ce57e80 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:11.879088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b019b4-8a45-4af6-8c8a-43053ce57e80 None admin] PATH_INFO: `/v3/services` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:11.879965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b019b4-8a45-4af6-8c8a-43053ce57e80 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:11.880263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b019b4-8a45-4af6-8c8a-43053ce57e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:11.881634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b019b4-8a45-4af6-8c8a-43053ce57e80 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:11.885898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 642/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 9 01:39:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:14.017997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d72728-88ff-4df6-a8b1-e371c41cb8ff None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:14.018540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d72728-88ff-4df6-a8b1-e371c41cb8ff None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:14.018540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d72728-88ff-4df6-a8b1-e371c41cb8ff None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:14.019862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 651/1293] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:39:14.024919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf533d-39b5-43b7-ad2c-49cddb40621d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:14.025324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf533d-39b5-43b7-ad2c-49cddb40621d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:14.025498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf533d-39b5-43b7-ad2c-49cddb40621d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:14.043951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ddcf533d-39b5-43b7-ad2c-49cddb40621d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:39:14.045164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ddcf533d-39b5-43b7-ad2c-49cddb40621d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:39:14.074785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:14.115265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 643/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:39:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:39:14.400300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d72728-88ff-4df6-a8b1-e371c41cb8ff None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:14.430686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:14.498587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d72728-88ff-4df6-a8b1-e371c41cb8ff None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:14.533720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762be391-b495-41c5-9439-afd874e91bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:14.534316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be391-b495-41c5-9439-afd874e91bbd None admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:14.534589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be391-b495-41c5-9439-afd874e91bbd None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:14.534854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762be391-b495-41c5-9439-afd874e91bbd None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:14.535498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be391-b495-41c5-9439-afd874e91bbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:14.536839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762be391-b495-41c5-9439-afd874e91bbd None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:14.550325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 652/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Fri Sep 9 01:39:14 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:39:16.680569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88f47-7ac9-43a9-b621-d59e427e4393 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:16.681151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88f47-7ac9-43a9-b621-d59e427e4393 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:16.681151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88f47-7ac9-43a9-b621-d59e427e4393 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:16.682465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 644/1296] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:39:16.687720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c4d2-f60a-4a78-812e-fe6a7c8c421c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:16.688146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c4d2-f60a-4a78-812e-fe6a7c8c421c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:16.688240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c4d2-f60a-4a78-812e-fe6a7c8c421c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:16.706955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-df84c4d2-f60a-4a78-812e-fe6a7c8c421c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:39:16.708121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-df84c4d2-f60a-4a78-812e-fe6a7c8c421c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:39:16.776724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 653/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:39:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:39:17.045124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb88f47-7ac9-43a9-b621-d59e427e4393 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:17.143568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb88f47-7ac9-43a9-b621-d59e427e4393 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:17.177012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:17.177653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:17.177929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:17.178200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] PATH_INFO: `/v3/services/placement` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:17.178771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:17.180122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:17.189169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:17.190313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:17.191224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 645/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:39:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:17.196170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84c4d2-f60a-4a78-812e-fe6a7c8c421c None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:17.214339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:17.291794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84c4d2-f60a-4a78-812e-fe6a7c8c421c None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:17.324603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:17.325340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:17.325614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:17.325881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:17.326490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:17.326786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:17.328152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:17.333079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 654/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 9 01:39:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:17.338869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:17.369193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:17.439237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9f81-4f6c-4fbf-935a-bfc4f7d4e76a None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:17.470642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf70742-b78c-4cbe-aebc-8a185a0e01d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:17.471229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70742-b78c-4cbe-aebc-8a185a0e01d8 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:17.471502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70742-b78c-4cbe-aebc-8a185a0e01d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:17.471767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf70742-b78c-4cbe-aebc-8a185a0e01d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:17.472448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf70742-b78c-4cbe-aebc-8a185a0e01d8 None admin] RBAC: Adding query filter params (interface=public, service_id=43c1d238dacf475397702cceb266542d, region_id=RegionOne) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:17.472748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf70742-b78c-4cbe-aebc-8a185a0e01d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:17.474123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf70742-b78c-4cbe-aebc-8a185a0e01d8 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:17.478991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 646/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Fri Sep 9 01:39:17 2022] GET /identity/v3/endpoints?service_id=43c1d238dacf475397702cceb266542d&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:17.485384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:17.562807 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:17.580778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ee93-abb3-4d87-a877-89a18d471762 None admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:17.612714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b25ee7-5392-446f-8543-4f04f68c6771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:17.613326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25ee7-5392-446f-8543-4f04f68c6771 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:17.613600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25ee7-5392-446f-8543-4f04f68c6771 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:17.613860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25ee7-5392-446f-8543-4f04f68c6771 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:17.614470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b25ee7-5392-446f-8543-4f04f68c6771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:17.615810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b25ee7-5392-446f-8543-4f04f68c6771 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:17.622276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 655/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Fri Sep 9 01:39:17 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:39:19.756014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34fa4-4371-4922-98dc-63e3a11aafef None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:19.756546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34fa4-4371-4922-98dc-63e3a11aafef None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:19.756546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34fa4-4371-4922-98dc-63e3a11aafef None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:19.757934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 647/1302] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:39:19.763171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acb3c8-e501-4924-b480-b1c03c387875 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:19.763585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acb3c8-e501-4924-b480-b1c03c387875 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:19.763696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acb3c8-e501-4924-b480-b1c03c387875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:19.782350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-01acb3c8-e501-4924-b480-b1c03c387875 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:39:19.783536 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-01acb3c8-e501-4924-b480-b1c03c387875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:39:19.847177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 656/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:39:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 01:39:20.117320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34fa4-4371-4922-98dc-63e3a11aafef None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:20.210692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:20.217555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf34fa4-4371-4922-98dc-63e3a11aafef None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:20.250408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:20.251016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:20.251290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:20.251555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:20.252128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:20.253499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 01:39:20.262208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 01:39:20.263367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application with Lock( Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self._enter() Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application return self.creator() Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application created_value = creator( Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: ERROR keystone.server.flask.application Sep 09 01:39:20.264420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 648/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Fri Sep 9 01:39:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 01:39:20.269436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acb3c8-e501-4924-b480-b1c03c387875 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:20.340938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:20.363743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acb3c8-e501-4924-b480-b1c03c387875 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:20.395911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b9396-0e12-443b-a901-c142d5b68f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:20.396441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b9396-0e12-443b-a901-c142d5b68f67 None admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:20.396713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b9396-0e12-443b-a901-c142d5b68f67 None admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:20.396980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b9396-0e12-443b-a901-c142d5b68f67 None admin] PATH_INFO: `/v3/services` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:20.397622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b9396-0e12-443b-a901-c142d5b68f67 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:39:20.397921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b9396-0e12-443b-a901-c142d5b68f67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:20.399252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b9396-0e12-443b-a901-c142d5b68f67 None admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:20.404092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 657/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Fri Sep 9 01:39:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:39:20.409801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:39:20.505848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eee4b4-1250-4529-a420-519831e94589 None admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:39:20.537394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3a3e2-acd4-4ab6-9989-24240d3b8003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:39:20.537974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3a3e2-acd4-4ab6-9989-24240d3b8003 None admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:39:20.538252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3a3e2-acd4-4ab6-9989-24240d3b8003 None admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:39:20.538521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3a3e2-acd4-4ab6-9989-24240d3b8003 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:39:20.539158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3a3e2-acd4-4ab6-9989-24240d3b8003 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:39:20.540485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee3a3e2-acd4-4ab6-9989-24240d3b8003 None admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:39:20.560414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 649/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:39:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:39:30.401294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=30 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:39:30.556135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:24.060416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542f03a-c8cf-40ce-ac92-241ae6d1c9f7 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:24.060896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542f03a-c8cf-40ce-ac92-241ae6d1c9f7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:24.060985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542f03a-c8cf-40ce-ac92-241ae6d1c9f7 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:24.066587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 658/1307] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:24.067579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe95cac-af62-4f5a-8752-174958dd1539 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:24.067986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe95cac-af62-4f5a-8752-174958dd1539 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:24.068080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe95cac-af62-4f5a-8752-174958dd1539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:24.208913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-dbe95cac-af62-4f5a-8752-174958dd1539 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:24.212389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-dbe95cac-af62-4f5a-8752-174958dd1539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:24.376213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 650/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:40:24.668964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070696-7608-49d7-a38d-5cf4e3da5ddd None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:24.669287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070696-7608-49d7-a38d-5cf4e3da5ddd None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:24.669566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c070696-7608-49d7-a38d-5cf4e3da5ddd None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:24.670921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 659/1309] 10.222.0.15 () {54 vars in 965 bytes} [Fri Sep 9 01:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:24.677432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a76c92-3171-48d0-ae5b-34eccf8e9eb8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:24.677718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a76c92-3171-48d0-ae5b-34eccf8e9eb8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:24.677992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a76c92-3171-48d0-ae5b-34eccf8e9eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:24.767048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-17a76c92-3171-48d0-ae5b-34eccf8e9eb8 None None] MFA Rules not processed for user `341277e7e3144e5ea93fa657916821e1`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:24.768428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-17a76c92-3171-48d0-ae5b-34eccf8e9eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:24.790737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:24.853210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 651/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Fri Sep 9 01:40:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:24.862116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c070696-7608-49d7-a38d-5cf4e3da5ddd None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:24.863841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0c070696-7608-49d7-a38d-5cf4e3da5ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:24.987947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c070696-7608-49d7-a38d-5cf4e3da5ddd None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:25.067699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c918014-ce39-49d0-b53e-2152553a84cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:25.068350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c918014-ce39-49d0-b53e-2152553a84cf service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:25.068632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c918014-ce39-49d0-b53e-2152553a84cf service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:25.068846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c918014-ce39-49d0-b53e-2152553a84cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:25.070438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7c918014-ce39-49d0-b53e-2152553a84cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:25.133187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:25.191880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c918014-ce39-49d0-b53e-2152553a84cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:25.193591 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c918014-ce39-49d0-b53e-2152553a84cf service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:25.214387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 660/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:27.698500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3f8ab-7d25-44d5-b595-584e5a6b161f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:27.699013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3f8ab-7d25-44d5-b595-584e5a6b161f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:27.699013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3f8ab-7d25-44d5-b595-584e5a6b161f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:27.700960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 652/1312] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:27.706271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd7b7-b18d-41cf-b873-9594b8f10f86 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:27.706693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd7b7-b18d-41cf-b873-9594b8f10f86 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:27.706938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fd7b7-b18d-41cf-b873-9594b8f10f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:27.800662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a43fd7b7-b18d-41cf-b873-9594b8f10f86 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:27.801912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a43fd7b7-b18d-41cf-b873-9594b8f10f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:27.900223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 661/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:28.181382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3f8ab-7d25-44d5-b595-584e5a6b161f None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:28.182896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-60d3f8ab-7d25-44d5-b595-584e5a6b161f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:28.287088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3f8ab-7d25-44d5-b595-584e5a6b161f None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:28.364000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:28.366540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd3734-cc26-4d06-aae7-921dfd0825db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:28.367360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd3734-cc26-4d06-aae7-921dfd0825db service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:28.367830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd3734-cc26-4d06-aae7-921dfd0825db service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:28.368150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd3734-cc26-4d06-aae7-921dfd0825db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:28.369958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-59bd3734-cc26-4d06-aae7-921dfd0825db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:28.478303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd3734-cc26-4d06-aae7-921dfd0825db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:28.479720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd3734-cc26-4d06-aae7-921dfd0825db service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:28.498823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 653/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:30.947126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d864d-9767-4daa-bb67-d2a8e589bbd4 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:30.947645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d864d-9767-4daa-bb67-d2a8e589bbd4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:30.947645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d864d-9767-4daa-bb67-d2a8e589bbd4 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:30.949004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 662/1315] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:30.954755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3428bf-6855-4e3f-b43f-428f2810c626 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:30.955161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3428bf-6855-4e3f-b43f-428f2810c626 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:30.955278 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3428bf-6855-4e3f-b43f-428f2810c626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:31.020148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-be3428bf-6855-4e3f-b43f-428f2810c626 None None] MFA Rules not processed for user `9190860e1df04a81b08ae447688ed74d`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:31.021368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-be3428bf-6855-4e3f-b43f-428f2810c626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:31.072302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:31.116615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 654/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:31.401013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d864d-9767-4daa-bb67-d2a8e589bbd4 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:31.402587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-444d864d-9767-4daa-bb67-d2a8e589bbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:31.412308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:31.504575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d864d-9767-4daa-bb67-d2a8e589bbd4 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:31.574903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a9ab4-7e2f-4c98-8559-565a0b5c9a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:31.575499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a9ab4-7e2f-4c98-8559-565a0b5c9a68 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:31.575770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a9ab4-7e2f-4c98-8559-565a0b5c9a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:31.576034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a9ab4-7e2f-4c98-8559-565a0b5c9a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:31.577599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-243a9ab4-7e2f-4c98-8559-565a0b5c9a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:31.637646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:31.683736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a9ab4-7e2f-4c98-8559-565a0b5c9a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:31.685718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a9ab4-7e2f-4c98-8559-565a0b5c9a68 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:31.703546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 663/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:34.378501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2677cf-5423-4b96-9f05-aaa52e909b4f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:34.379309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2677cf-5423-4b96-9f05-aaa52e909b4f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:34.379309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2677cf-5423-4b96-9f05-aaa52e909b4f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:34.380649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 655/1318] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:34.385898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660eddb-bc39-4476-bef1-4158e34d5b7c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:34.386325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660eddb-bc39-4476-bef1-4158e34d5b7c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:34.386421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660eddb-bc39-4476-bef1-4158e34d5b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:34.451056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6660eddb-bc39-4476-bef1-4158e34d5b7c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:34.452246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6660eddb-bc39-4476-bef1-4158e34d5b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:34.544999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 664/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:40:34.830910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2677cf-5423-4b96-9f05-aaa52e909b4f None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:34.832459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2a2677cf-5423-4b96-9f05-aaa52e909b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:34.934575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2677cf-5423-4b96-9f05-aaa52e909b4f None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:35.009785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b7a32-aeb4-4cc3-ae1d-ffc85ecd33f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:35.010384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7a32-aeb4-4cc3-ae1d-ffc85ecd33f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:35.010662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7a32-aeb4-4cc3-ae1d-ffc85ecd33f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:35.010931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b7a32-aeb4-4cc3-ae1d-ffc85ecd33f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:35.012541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cb6b7a32-aeb4-4cc3-ae1d-ffc85ecd33f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:35.019437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:35.122565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b7a32-aeb4-4cc3-ae1d-ffc85ecd33f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:35.123906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b7a32-aeb4-4cc3-ae1d-ffc85ecd33f8 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:35.141788 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 656/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:37.939711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd21695-0afd-4dc8-bc6c-a09bab284252 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:37.940274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd21695-0afd-4dc8-bc6c-a09bab284252 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:37.940274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd21695-0afd-4dc8-bc6c-a09bab284252 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:37.941657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 665/1321] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:37.946975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69167-f784-4a7d-9024-5115b9b4e363 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:37.947382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69167-f784-4a7d-9024-5115b9b4e363 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:37.947682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69167-f784-4a7d-9024-5115b9b4e363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:38.022224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7bf69167-f784-4a7d-9024-5115b9b4e363 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:38.023557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7bf69167-f784-4a7d-9024-5115b9b4e363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:38.085418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:38.128290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 657/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:38.409490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd21695-0afd-4dc8-bc6c-a09bab284252 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:38.411043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-acd21695-0afd-4dc8-bc6c-a09bab284252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:38.431909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:38.515707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd21695-0afd-4dc8-bc6c-a09bab284252 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:38.589294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77867c75-2ed7-4fcd-b1eb-51be7c6eb256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:38.589934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867c75-2ed7-4fcd-b1eb-51be7c6eb256 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:38.590204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867c75-2ed7-4fcd-b1eb-51be7c6eb256 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:38.590470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77867c75-2ed7-4fcd-b1eb-51be7c6eb256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:38.592018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-77867c75-2ed7-4fcd-b1eb-51be7c6eb256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:38.661777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:38.699839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77867c75-2ed7-4fcd-b1eb-51be7c6eb256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:38.701281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77867c75-2ed7-4fcd-b1eb-51be7c6eb256 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:38.719021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 666/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:40.907595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0834-12eb-44cb-9fec-764c3075c6fb None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:40.908078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0834-12eb-44cb-9fec-764c3075c6fb None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:40.908180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f0834-12eb-44cb-9fec-764c3075c6fb None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:40.909753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 658/1324] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:40.914723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c631-fef9-4277-918b-ce6901e9fa8d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:40.915142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c631-fef9-4277-918b-ce6901e9fa8d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:40.915284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328c631-fef9-4277-918b-ce6901e9fa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:40.979541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7328c631-fef9-4277-918b-ce6901e9fa8d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:40.980748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7328c631-fef9-4277-918b-ce6901e9fa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:41.084541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 667/1325] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:41.367978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0834-12eb-44cb-9fec-764c3075c6fb None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:41.369527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-143f0834-12eb-44cb-9fec-764c3075c6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:41.473117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f0834-12eb-44cb-9fec-764c3075c6fb None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:41.546608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e8220-d660-47c7-84ed-749154d352f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:41.547205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8220-d660-47c7-84ed-749154d352f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:41.547479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8220-d660-47c7-84ed-749154d352f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:41.547745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e8220-d660-47c7-84ed-749154d352f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:41.549324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ab9e8220-d660-47c7-84ed-749154d352f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:41.566328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:41.669530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e8220-d660-47c7-84ed-749154d352f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:41.670900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e8220-d660-47c7-84ed-749154d352f7 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:41.691208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 659/1326] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:43.912104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928430-db6f-4538-967e-4e3b16da9cd4 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:43.912539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928430-db6f-4538-967e-4e3b16da9cd4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:43.912654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f928430-db6f-4538-967e-4e3b16da9cd4 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:43.914041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 668/1327] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:43.919487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125faa-94f4-4283-9097-86e9423b3171 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:43.919942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125faa-94f4-4283-9097-86e9423b3171 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:43.920047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7125faa-94f4-4283-9097-86e9423b3171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:43.986761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d7125faa-94f4-4283-9097-86e9423b3171 None None] MFA Rules not processed for user `9190860e1df04a81b08ae447688ed74d`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:43.987895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d7125faa-94f4-4283-9097-86e9423b3171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:44.054536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:44.087836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 660/1328] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:44.369320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f928430-db6f-4538-967e-4e3b16da9cd4 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:44.370964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4f928430-db6f-4538-967e-4e3b16da9cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:44.404597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:44.482275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f928430-db6f-4538-967e-4e3b16da9cd4 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:44.558516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc51c73-821b-4249-b532-33dffa69262e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:44.559124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc51c73-821b-4249-b532-33dffa69262e service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:44.559408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc51c73-821b-4249-b532-33dffa69262e service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:44.559677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc51c73-821b-4249-b532-33dffa69262e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:44.561337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8bc51c73-821b-4249-b532-33dffa69262e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:44.639216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:44.670861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc51c73-821b-4249-b532-33dffa69262e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:44.672282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc51c73-821b-4249-b532-33dffa69262e service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:44.690661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 669/1329] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:47.075592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78374b-43e7-478d-b61d-3a98800f38f0 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:47.076152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78374b-43e7-478d-b61d-3a98800f38f0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:47.076152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c78374b-43e7-478d-b61d-3a98800f38f0 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:47.077533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 661/1330] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:47.083043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901357-5076-4488-a6ea-397d2fc2a553 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:47.083443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901357-5076-4488-a6ea-397d2fc2a553 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:47.083584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901357-5076-4488-a6ea-397d2fc2a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:47.151069 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-97901357-5076-4488-a6ea-397d2fc2a553 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:47.152181 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-97901357-5076-4488-a6ea-397d2fc2a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:47.245024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 670/1331] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:47.527421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78374b-43e7-478d-b61d-3a98800f38f0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:47.528933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2c78374b-43e7-478d-b61d-3a98800f38f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:47.638218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c78374b-43e7-478d-b61d-3a98800f38f0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:47.709615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027480-f9b7-4c43-a586-fe69c631902a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:47.710141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027480-f9b7-4c43-a586-fe69c631902a service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:47.710410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027480-f9b7-4c43-a586-fe69c631902a service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:47.710668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027480-f9b7-4c43-a586-fe69c631902a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:47.712218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-15027480-f9b7-4c43-a586-fe69c631902a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:47.738233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:47.819104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15027480-f9b7-4c43-a586-fe69c631902a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:47.820532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15027480-f9b7-4c43-a586-fe69c631902a service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:47.838066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 662/1332] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:50.056992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852b021-8ff2-494f-a63c-5930b4e05087 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:50.057566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852b021-8ff2-494f-a63c-5930b4e05087 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:50.057566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852b021-8ff2-494f-a63c-5930b4e05087 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:50.059453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 671/1333] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:50.064697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7e779-650b-4113-bab9-d85f4885d578 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:50.065147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7e779-650b-4113-bab9-d85f4885d578 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:50.065284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7e779-650b-4113-bab9-d85f4885d578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:50.133421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-65e7e779-650b-4113-bab9-d85f4885d578 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:50.134625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-65e7e779-650b-4113-bab9-d85f4885d578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:50.206382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:50.228493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 663/1334] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:50.512073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852b021-8ff2-494f-a63c-5930b4e05087 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:50.513619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3852b021-8ff2-494f-a63c-5930b4e05087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:50.552727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:50.614646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852b021-8ff2-494f-a63c-5930b4e05087 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:50.687658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ae653-8d91-4722-a180-b126d2483f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:50.688244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae653-8d91-4722-a180-b126d2483f90 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:50.688518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae653-8d91-4722-a180-b126d2483f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:50.688785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae653-8d91-4722-a180-b126d2483f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:50.690361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b44ae653-8d91-4722-a180-b126d2483f90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:50.772101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:50.797835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ae653-8d91-4722-a180-b126d2483f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:50.799251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ae653-8d91-4722-a180-b126d2483f90 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:50.817081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 672/1335] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:40:53.291022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a03eb-d1fb-4916-863c-a80f76dfc763 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:53.291585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a03eb-d1fb-4916-863c-a80f76dfc763 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:53.291585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a03eb-d1fb-4916-863c-a80f76dfc763 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:53.293457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 664/1336] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:53.298927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de007721-543f-4386-9ad2-13b739c2cf79 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:53.299339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de007721-543f-4386-9ad2-13b739c2cf79 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:53.299475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de007721-543f-4386-9ad2-13b739c2cf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:53.363636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-de007721-543f-4386-9ad2-13b739c2cf79 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:53.364824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-de007721-543f-4386-9ad2-13b739c2cf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:53.459706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 673/1337] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:53.742443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a03eb-d1fb-4916-863c-a80f76dfc763 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:53.743964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3b4a03eb-d1fb-4916-863c-a80f76dfc763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:53.844903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a03eb-d1fb-4916-863c-a80f76dfc763 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:53.917807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de177bd-597f-437d-9310-728a01b9b020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:53.918397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de177bd-597f-437d-9310-728a01b9b020 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:53.918668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de177bd-597f-437d-9310-728a01b9b020 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:53.918929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de177bd-597f-437d-9310-728a01b9b020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:53.920492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3de177bd-597f-437d-9310-728a01b9b020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:53.954984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:54.026239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de177bd-597f-437d-9310-728a01b9b020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:54.028117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de177bd-597f-437d-9310-728a01b9b020 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:54.045889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 665/1338] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:40:58.080680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b21f-ae90-4bbc-bcc9-6c2ce0a43948 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:58.081198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b21f-ae90-4bbc-bcc9-6c2ce0a43948 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:58.081198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b21f-ae90-4bbc-bcc9-6c2ce0a43948 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:58.082501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 674/1339] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:40:58.087650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58269ad4-39e4-49c1-90b9-6b375e27cca3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:58.088109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58269ad4-39e4-49c1-90b9-6b375e27cca3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:58.088205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58269ad4-39e4-49c1-90b9-6b375e27cca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:58.154183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-58269ad4-39e4-49c1-90b9-6b375e27cca3 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:40:58.155287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-58269ad4-39e4-49c1-90b9-6b375e27cca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:58.239619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:58.251929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 666/1340] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:40:58.545356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1b21f-ae90-4bbc-bcc9-6c2ce0a43948 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:40:58.547283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b0d1b21f-ae90-4bbc-bcc9-6c2ce0a43948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:58.596371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:58.652398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1b21f-ae90-4bbc-bcc9-6c2ce0a43948 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:40:58.731891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176261dd-108f-4fd2-bde4-b6ab82b90f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:40:58.732497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176261dd-108f-4fd2-bde4-b6ab82b90f4c service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:40:58.732771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176261dd-108f-4fd2-bde4-b6ab82b90f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:40:58.733037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176261dd-108f-4fd2-bde4-b6ab82b90f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:40:58.734629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-176261dd-108f-4fd2-bde4-b6ab82b90f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:40:58.827338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:40:58.844251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176261dd-108f-4fd2-bde4-b6ab82b90f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:40:58.845729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176261dd-108f-4fd2-bde4-b6ab82b90f4c service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:40:58.864053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 675/1341] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:01.656136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876801d6-0942-40fc-bb4d-eddf2ef38982 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:01.656704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876801d6-0942-40fc-bb4d-eddf2ef38982 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:01.656704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876801d6-0942-40fc-bb4d-eddf2ef38982 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:01.658100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 667/1342] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:01.664210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a575fb-d7c9-4f06-ba09-237ff2cfda2e None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:01.664735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a575fb-d7c9-4f06-ba09-237ff2cfda2e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:01.664735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a575fb-d7c9-4f06-ba09-237ff2cfda2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:01.730308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-99a575fb-d7c9-4f06-ba09-237ff2cfda2e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:01.731519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-99a575fb-d7c9-4f06-ba09-237ff2cfda2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:01.827703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 676/1343] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:41:02.113423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876801d6-0942-40fc-bb4d-eddf2ef38982 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:02.114945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-876801d6-0942-40fc-bb4d-eddf2ef38982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:02.223803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876801d6-0942-40fc-bb4d-eddf2ef38982 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:02.301158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b6928-8d31-4c3e-90c1-ac1123da1592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:02.301762 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6928-8d31-4c3e-90c1-ac1123da1592 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:02.302033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6928-8d31-4c3e-90c1-ac1123da1592 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:02.302306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6928-8d31-4c3e-90c1-ac1123da1592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:02.303876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-890b6928-8d31-4c3e-90c1-ac1123da1592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:02.347033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:02.414600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b6928-8d31-4c3e-90c1-ac1123da1592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:02.416067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b6928-8d31-4c3e-90c1-ac1123da1592 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:02.436486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 668/1344] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:06.592758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d4f3-7831-4156-8948-fb7ce83718af None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:06.593298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d4f3-7831-4156-8948-fb7ce83718af None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:06.593481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d4f3-7831-4156-8948-fb7ce83718af None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:06.594904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 677/1345] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:06.600328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a018e71-0d21-4254-a4cf-75f4db6d0aa2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:06.600720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a018e71-0d21-4254-a4cf-75f4db6d0aa2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:06.600966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a018e71-0d21-4254-a4cf-75f4db6d0aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:06.669631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1a018e71-0d21-4254-a4cf-75f4db6d0aa2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:06.670750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1a018e71-0d21-4254-a4cf-75f4db6d0aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:06.763218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:06.765554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 669/1346] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:07.044984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298d4f3-7831-4156-8948-fb7ce83718af None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:07.046568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7298d4f3-7831-4156-8948-fb7ce83718af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:07.102000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:07.148490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7298d4f3-7831-4156-8948-fb7ce83718af None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:07.219263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2bab6b-6f7a-49c4-bb3f-184e4b8a7173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:07.220109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bab6b-6f7a-49c4-bb3f-184e4b8a7173 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:07.220387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bab6b-6f7a-49c4-bb3f-184e4b8a7173 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:07.220654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bab6b-6f7a-49c4-bb3f-184e4b8a7173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:07.222228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2d2bab6b-6f7a-49c4-bb3f-184e4b8a7173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:07.326718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:07.330999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bab6b-6f7a-49c4-bb3f-184e4b8a7173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:07.332425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bab6b-6f7a-49c4-bb3f-184e4b8a7173 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:07.349939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 678/1347] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:12.293047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c34c6d-8299-4354-804f-82613c06fc24 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:12.293531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c34c6d-8299-4354-804f-82613c06fc24 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:12.293636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c34c6d-8299-4354-804f-82613c06fc24 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:12.295424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 670/1348] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:41:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:12.300400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a7f2-6d06-42c1-bcc6-be606554d769 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:12.300877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a7f2-6d06-42c1-bcc6-be606554d769 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:12.300983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4a7f2-6d06-42c1-bcc6-be606554d769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:12.446917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-83f4a7f2-6d06-42c1-bcc6-be606554d769 None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:12.448129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-83f4a7f2-6d06-42c1-bcc6-be606554d769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:12.568855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 679/1349] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:41:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:16.768623 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598a09d-0c4d-4754-af36-eb6cc7921385 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:16.769529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598a09d-0c4d-4754-af36-eb6cc7921385 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:16.769986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598a09d-0c4d-4754-af36-eb6cc7921385 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:16.771783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 671/1350] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:16.777231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c715a-0462-45d6-a190-8ca6620c75c3 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:16.777651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c715a-0462-45d6-a190-8ca6620c75c3 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:16.777886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c715a-0462-45d6-a190-8ca6620c75c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:16.833275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:16.859318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5a2c715a-0462-45d6-a190-8ca6620c75c3 None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:16.860527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5a2c715a-0462-45d6-a190-8ca6620c75c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:16.981758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 680/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:41:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:18.264950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6df68-97f2-46e0-a2de-38e3cd6882cf None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:18.265509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6df68-97f2-46e0-a2de-38e3cd6882cf None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:18.265509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6df68-97f2-46e0-a2de-38e3cd6882cf None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:18.266803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 672/1352] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:18.272019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddc987-6d11-473d-94be-2033eca7460f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:18.272485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddc987-6d11-473d-94be-2033eca7460f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:18.272651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ddc987-6d11-473d-94be-2033eca7460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:18.349579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d6ddc987-6d11-473d-94be-2033eca7460f None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:18.350781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d6ddc987-6d11-473d-94be-2033eca7460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:18.393326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:18.457979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 681/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:41:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:22.036358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1659a-7cb2-4a8b-8b9b-9822896d61fd None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:22.036864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1659a-7cb2-4a8b-8b9b-9822896d61fd None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:22.036864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1659a-7cb2-4a8b-8b9b-9822896d61fd None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:22.038254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 673/1354] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:22.044557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51afc1-fd26-4f36-a403-7c78f2a30936 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:22.045094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51afc1-fd26-4f36-a403-7c78f2a30936 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:22.045209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51afc1-fd26-4f36-a403-7c78f2a30936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:22.113940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-de51afc1-fd26-4f36-a403-7c78f2a30936 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:22.115369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-de51afc1-fd26-4f36-a403-7c78f2a30936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:22.220247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 682/1355] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:41:22.520506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87a948-8879-4c4b-8cef-1cfb9e368236 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:22.520798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87a948-8879-4c4b-8cef-1cfb9e368236 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:22.521096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87a948-8879-4c4b-8cef-1cfb9e368236 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:22.522751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 674/1356] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 9 01:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:22.529834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff20368-be61-4c22-b628-a31df2022567 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:22.530125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff20368-be61-4c22-b628-a31df2022567 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:22.530409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff20368-be61-4c22-b628-a31df2022567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:22.547496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:22.631060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-cff20368-be61-4c22-b628-a31df2022567 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:22.632516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-cff20368-be61-4c22-b628-a31df2022567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:22.753579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 683/1357] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 9 01:41:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:41:22.761801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87a948-8879-4c4b-8cef-1cfb9e368236 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:22.763344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ce87a948-8879-4c4b-8cef-1cfb9e368236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:22.924126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87a948-8879-4c4b-8cef-1cfb9e368236 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:23.002682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:23.050067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe76e0-aeaa-4f58-bce4-7c123977c0d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:23.050883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe76e0-aeaa-4f58-bce4-7c123977c0d8 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:23.051336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe76e0-aeaa-4f58-bce4-7c123977c0d8 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:23.051757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe76e0-aeaa-4f58-bce4-7c123977c0d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:23.053570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7fbe76e0-aeaa-4f58-bce4-7c123977c0d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:23.156413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe76e0-aeaa-4f58-bce4-7c123977c0d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:23.158049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe76e0-aeaa-4f58-bce4-7c123977c0d8 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:23.179138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 675/1358] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:25.548344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c9429-dd19-4810-a9c3-0bd372b7fc05 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:25.548871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c9429-dd19-4810-a9c3-0bd372b7fc05 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:25.548871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c9429-dd19-4810-a9c3-0bd372b7fc05 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:25.550296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 684/1359] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:25.555711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198110a5-dd09-4cd8-aacc-2dc2ba1440e5 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:25.556117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198110a5-dd09-4cd8-aacc-2dc2ba1440e5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:25.556117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198110a5-dd09-4cd8-aacc-2dc2ba1440e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:25.625702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-198110a5-dd09-4cd8-aacc-2dc2ba1440e5 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:25.626905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-198110a5-dd09-4cd8-aacc-2dc2ba1440e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:25.667329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:25.746060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 676/1360] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:26.034475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c9429-dd19-4810-a9c3-0bd372b7fc05 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:26.035974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-eb1c9429-dd19-4810-a9c3-0bd372b7fc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:26.105094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:26.203746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c9429-dd19-4810-a9c3-0bd372b7fc05 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:26.354765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbb4c0-0438-4336-a139-6fcd40c38464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:26.355305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb4c0-0438-4336-a139-6fcd40c38464 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:26.355755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb4c0-0438-4336-a139-6fcd40c38464 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:26.356085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbb4c0-0438-4336-a139-6fcd40c38464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:26.358262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-10cbb4c0-0438-4336-a139-6fcd40c38464 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:26.381708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:26.451369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbb4c0-0438-4336-a139-6fcd40c38464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:26.452978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbb4c0-0438-4336-a139-6fcd40c38464 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:26.471326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 685/1361] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:28.959896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e79e1-be60-4350-bec0-e94bfb5adf8f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:28.960417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e79e1-be60-4350-bec0-e94bfb5adf8f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:28.960417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e79e1-be60-4350-bec0-e94bfb5adf8f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:28.963869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 677/1362] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:28.970600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a570-33bd-4246-96e7-4d27c38da306 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:28.971128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a570-33bd-4246-96e7-4d27c38da306 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:28.971361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11a570-33bd-4246-96e7-4d27c38da306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:29.043643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7b11a570-33bd-4246-96e7-4d27c38da306 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:29.045073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7b11a570-33bd-4246-96e7-4d27c38da306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:29.143868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:29.153125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 686/1363] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:29.440192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edf142-c52d-4736-8402-38d7f5431eb7 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:29.440677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edf142-c52d-4736-8402-38d7f5431eb7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:29.441138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edf142-c52d-4736-8402-38d7f5431eb7 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:29.443171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 678/1364] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 9 01:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:29.449377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d85863-04c7-426d-bdb1-ac5d9bc47d32 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:29.449659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d85863-04c7-426d-bdb1-ac5d9bc47d32 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:29.449932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d85863-04c7-426d-bdb1-ac5d9bc47d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:29.529795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a4d85863-04c7-426d-bdb1-ac5d9bc47d32 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:29.531111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a4d85863-04c7-426d-bdb1-ac5d9bc47d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:29.666115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 687/1365] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 9 01:41:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:29.674330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edf142-c52d-4736-8402-38d7f5431eb7 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:29.675867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-49edf142-c52d-4736-8402-38d7f5431eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:29.828311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edf142-c52d-4736-8402-38d7f5431eb7 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:29.850668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:29.957892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b244ecd-e32d-49c9-8929-d0369d4be249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:29.958499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b244ecd-e32d-49c9-8929-d0369d4be249 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:29.958781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b244ecd-e32d-49c9-8929-d0369d4be249 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:29.959038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b244ecd-e32d-49c9-8929-d0369d4be249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:29.960609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0b244ecd-e32d-49c9-8929-d0369d4be249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:30.063915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b244ecd-e32d-49c9-8929-d0369d4be249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:30.065565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b244ecd-e32d-49c9-8929-d0369d4be249 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:30.084554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 679/1366] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:30.273229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057ab0b-860e-4518-bf7a-6f854aa47b74 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:30.273788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057ab0b-860e-4518-bf7a-6f854aa47b74 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:30.273788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057ab0b-860e-4518-bf7a-6f854aa47b74 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:30.275625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 688/1367] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:30.281258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49089ca-bb3e-46b5-a495-be9f93bfc2fb None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:30.281577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49089ca-bb3e-46b5-a495-be9f93bfc2fb None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:30.281855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49089ca-bb3e-46b5-a495-be9f93bfc2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:30.317731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:30.362616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a49089ca-bb3e-46b5-a495-be9f93bfc2fb None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:30.364304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a49089ca-bb3e-46b5-a495-be9f93bfc2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:30.468660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 680/1368] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:30.801894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2aae3-63f5-4010-ac6c-9bb12106afd2 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:30.802177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2aae3-63f5-4010-ac6c-9bb12106afd2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:30.802446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2aae3-63f5-4010-ac6c-9bb12106afd2 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:30.803761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 689/1369] 10.222.0.15 () {54 vars in 964 bytes} [Fri Sep 9 01:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:30.808713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd81f7-52ed-4ab0-8ff7-60556c476e2c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:30.809139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd81f7-52ed-4ab0-8ff7-60556c476e2c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:30.809514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd81f7-52ed-4ab0-8ff7-60556c476e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:30.891120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b6bd81f7-52ed-4ab0-8ff7-60556c476e2c None None] MFA Rules not processed for user `df27a2ce80b94237bd2b9ecfe35109cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:30.893632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b6bd81f7-52ed-4ab0-8ff7-60556c476e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:30.947601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:30.979420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 681/1370] 10.222.0.15 () {58 vars in 1061 bytes} [Fri Sep 9 01:41:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:30.987826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2aae3-63f5-4010-ac6c-9bb12106afd2 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:30.989478 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0ab2aae3-63f5-4010-ac6c-9bb12106afd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:31.042762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:31.101853 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2aae3-63f5-4010-ac6c-9bb12106afd2 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:31.181847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaceb1a8-f55e-4e92-ac8e-c61815577141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:31.182686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaceb1a8-f55e-4e92-ac8e-c61815577141 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:31.183116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaceb1a8-f55e-4e92-ac8e-c61815577141 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:31.183511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaceb1a8-f55e-4e92-ac8e-c61815577141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:31.185405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-eaceb1a8-f55e-4e92-ac8e-c61815577141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:31.325838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:31.346104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaceb1a8-f55e-4e92-ac8e-c61815577141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:31.348132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaceb1a8-f55e-4e92-ac8e-c61815577141 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:31.375196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 690/1371] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:32.347283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5560439-eac7-4158-9c16-762e6d3241d5 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:32.347819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5560439-eac7-4158-9c16-762e6d3241d5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:32.347819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5560439-eac7-4158-9c16-762e6d3241d5 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:32.349442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 682/1372] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:32.355616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37741503-8cd2-4cbe-96c3-f8d18ee8b9ed None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:32.356108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37741503-8cd2-4cbe-96c3-f8d18ee8b9ed None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:32.356291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37741503-8cd2-4cbe-96c3-f8d18ee8b9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:32.422650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-37741503-8cd2-4cbe-96c3-f8d18ee8b9ed None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:32.423858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-37741503-8cd2-4cbe-96c3-f8d18ee8b9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:32.522883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 691/1373] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:32.817881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5560439-eac7-4158-9c16-762e6d3241d5 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:32.819385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a5560439-eac7-4158-9c16-762e6d3241d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:32.989376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5560439-eac7-4158-9c16-762e6d3241d5 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:33.048161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:33.116956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828837d4-b8ca-44fd-9315-779cfd36e943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:33.117592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828837d4-b8ca-44fd-9315-779cfd36e943 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:33.117871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828837d4-b8ca-44fd-9315-779cfd36e943 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:33.118141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828837d4-b8ca-44fd-9315-779cfd36e943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:33.119738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-828837d4-b8ca-44fd-9315-779cfd36e943 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:33.212326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828837d4-b8ca-44fd-9315-779cfd36e943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:33.214085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828837d4-b8ca-44fd-9315-779cfd36e943 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:33.232694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 683/1374] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:35.467968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595273-14de-477e-b23f-fe8f1e1bcf1c None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:35.468468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595273-14de-477e-b23f-fe8f1e1bcf1c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:35.468581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38595273-14de-477e-b23f-fe8f1e1bcf1c None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:35.470065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 692/1375] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:35.475208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a0c-0065-4ad1-b8d8-dc2cf7b52f08 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:35.475630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a0c-0065-4ad1-b8d8-dc2cf7b52f08 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:35.475840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c0a0c-0065-4ad1-b8d8-dc2cf7b52f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:35.544863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-218c0a0c-0065-4ad1-b8d8-dc2cf7b52f08 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:35.546103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-218c0a0c-0065-4ad1-b8d8-dc2cf7b52f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:35.553570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:35.654887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 684/1376] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:36.086336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38595273-14de-477e-b23f-fe8f1e1bcf1c None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:36.088556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-38595273-14de-477e-b23f-fe8f1e1bcf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:36.158587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:36.252214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38595273-14de-477e-b23f-fe8f1e1bcf1c None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:36.394963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fdd8e5-ae8f-4b0d-a3f8-808b284eb41a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:36.395617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdd8e5-ae8f-4b0d-a3f8-808b284eb41a service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:36.395894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdd8e5-ae8f-4b0d-a3f8-808b284eb41a service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:36.396193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdd8e5-ae8f-4b0d-a3f8-808b284eb41a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:36.397938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-48fdd8e5-ae8f-4b0d-a3f8-808b284eb41a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:36.421116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:36.504276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdd8e5-ae8f-4b0d-a3f8-808b284eb41a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:36.506076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fdd8e5-ae8f-4b0d-a3f8-808b284eb41a service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:36.536680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 693/1377] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:41:38.818806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb316a66-006d-4a4a-81d3-a693113cfb5e None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:38.819236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb316a66-006d-4a4a-81d3-a693113cfb5e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:38.819485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb316a66-006d-4a4a-81d3-a693113cfb5e None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:38.825114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 685/1378] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:38.826568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb00a6-47b0-49b9-bc0d-1bc9b514019d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:38.826948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb00a6-47b0-49b9-bc0d-1bc9b514019d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:38.827226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb00a6-47b0-49b9-bc0d-1bc9b514019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:38.898908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f3fb00a6-47b0-49b9-bc0d-1bc9b514019d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:38.898908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f3fb00a6-47b0-49b9-bc0d-1bc9b514019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:39.000449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:39.007282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 694/1379] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:41:39.298450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb316a66-006d-4a4a-81d3-a693113cfb5e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:39.300058 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-eb316a66-006d-4a4a-81d3-a693113cfb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:39.451146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:39.462341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb316a66-006d-4a4a-81d3-a693113cfb5e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:39.593163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b9b86-3fc1-4c70-84f4-8277449a57d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:39.593976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9b86-3fc1-4c70-84f4-8277449a57d9 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:39.594323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9b86-3fc1-4c70-84f4-8277449a57d9 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:39.594691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b9b86-3fc1-4c70-84f4-8277449a57d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:39.596718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0b6b9b86-3fc1-4c70-84f4-8277449a57d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:39.707075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b9b86-3fc1-4c70-84f4-8277449a57d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:39.709186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b9b86-3fc1-4c70-84f4-8277449a57d9 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:39.722787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:39.732956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 686/1380] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:41:41.990331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49de93-8723-470b-9086-05c755616cd2 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:41.991304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49de93-8723-470b-9086-05c755616cd2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:41.991774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49de93-8723-470b-9086-05c755616cd2 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:41.993397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 695/1381] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:41.999923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acc9b-4d10-4582-a0c5-94f05bdfc872 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:42.000497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acc9b-4d10-4582-a0c5-94f05bdfc872 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:42.001421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642acc9b-4d10-4582-a0c5-94f05bdfc872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:42.079561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-642acc9b-4d10-4582-a0c5-94f05bdfc872 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:42.080784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-642acc9b-4d10-4582-a0c5-94f05bdfc872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:42.189710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 687/1382] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:41:42.480765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49de93-8723-470b-9086-05c755616cd2 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:42.482785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-dc49de93-8723-470b-9086-05c755616cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:42.651638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49de93-8723-470b-9086-05c755616cd2 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:42.718402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:42.765594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c101cd4-4506-414e-9914-1f814a9a4843 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:42.766206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c101cd4-4506-414e-9914-1f814a9a4843 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:42.766486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c101cd4-4506-414e-9914-1f814a9a4843 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:42.766762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c101cd4-4506-414e-9914-1f814a9a4843 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:42.768366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9c101cd4-4506-414e-9914-1f814a9a4843 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:42.859698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c101cd4-4506-414e-9914-1f814a9a4843 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:42.861336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c101cd4-4506-414e-9914-1f814a9a4843 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:42.879382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 696/1383] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:45.092112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ccf8b-cbbf-460c-ba20-07cd16863154 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:45.092664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ccf8b-cbbf-460c-ba20-07cd16863154 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:45.092664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ccf8b-cbbf-460c-ba20-07cd16863154 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:45.094187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 688/1384] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:45.099192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf44d99-f8ac-40c1-aba6-05dfdda938b8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:45.099632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf44d99-f8ac-40c1-aba6-05dfdda938b8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:45.099727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf44d99-f8ac-40c1-aba6-05dfdda938b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:45.166882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4cf44d99-f8ac-40c1-aba6-05dfdda938b8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:45.167986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4cf44d99-f8ac-40c1-aba6-05dfdda938b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:45.201369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:45.266690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 697/1385] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:41:45.557647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ccf8b-cbbf-460c-ba20-07cd16863154 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:45.559169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7a5ccf8b-cbbf-460c-ba20-07cd16863154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:45.634953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:45.706215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ccf8b-cbbf-460c-ba20-07cd16863154 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:45.829245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9473f-272c-4084-96dd-176915078fad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:45.829857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9473f-272c-4084-96dd-176915078fad service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:45.830248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9473f-272c-4084-96dd-176915078fad service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:45.830529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9473f-272c-4084-96dd-176915078fad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:45.832176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3cc9473f-272c-4084-96dd-176915078fad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:45.872305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:45.928685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9473f-272c-4084-96dd-176915078fad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:45.930307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9473f-272c-4084-96dd-176915078fad service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:45.948425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 689/1386] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:46.714731 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285f925-dde6-4d39-bcc8-d5a129361dc9 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:46.715273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285f925-dde6-4d39-bcc8-d5a129361dc9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:46.715273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285f925-dde6-4d39-bcc8-d5a129361dc9 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:46.716602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 698/1387] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:46.722008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444317-01f5-468e-996b-dcc3539ad29c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:46.722405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444317-01f5-468e-996b-dcc3539ad29c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:46.722627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444317-01f5-468e-996b-dcc3539ad29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:46.788533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-65444317-01f5-468e-996b-dcc3539ad29c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:46.790247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-65444317-01f5-468e-996b-dcc3539ad29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:46.909183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 690/1388] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:47.208505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b89296-c387-4a21-8062-a3cb9b333d47 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:47.208878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b89296-c387-4a21-8062-a3cb9b333d47 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:47.209248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b89296-c387-4a21-8062-a3cb9b333d47 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:47.210638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 699/1389] 10.222.0.15 () {54 vars in 966 bytes} [Fri Sep 9 01:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:47.215999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be369f-fb5a-4c26-8edb-e3980d2b8441 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:47.216291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be369f-fb5a-4c26-8edb-e3980d2b8441 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:47.216573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be369f-fb5a-4c26-8edb-e3980d2b8441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:47.232349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:47.294824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b5be369f-fb5a-4c26-8edb-e3980d2b8441 None None] MFA Rules not processed for user `029447e782484e8f8a704b683ede1819`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:47.296008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b5be369f-fb5a-4c26-8edb-e3980d2b8441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:47.371337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 691/1390] 10.222.0.15 () {58 vars in 1063 bytes} [Fri Sep 9 01:41:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:47.380282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b89296-c387-4a21-8062-a3cb9b333d47 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:47.381799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-72b89296-c387-4a21-8062-a3cb9b333d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:47.486516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b89296-c387-4a21-8062-a3cb9b333d47 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:47.545230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:47.561556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40483601-1f1f-406e-bc52-912b624c9457 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029447e782484e8f8a704b683ede1819', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:47.562158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483601-1f1f-406e-bc52-912b624c9457 service glance] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:47.562434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483601-1f1f-406e-bc52-912b624c9457 service glance] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:47.562699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483601-1f1f-406e-bc52-912b624c9457 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:47.564244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-40483601-1f1f-406e-bc52-912b624c9457 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:47.673458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40483601-1f1f-406e-bc52-912b624c9457 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:47.674907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40483601-1f1f-406e-bc52-912b624c9457 service glance] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:47.692991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 700/1391] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:48.155720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f032ff-3bbf-4205-8695-ba6a02b23dc3 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:48.156760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f032ff-3bbf-4205-8695-ba6a02b23dc3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:48.157409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f032ff-3bbf-4205-8695-ba6a02b23dc3 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:48.159444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 692/1392] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:48.166789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fba6a-a612-4cd0-9098-31ef62577497 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:48.167737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fba6a-a612-4cd0-9098-31ef62577497 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:48.168123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fba6a-a612-4cd0-9098-31ef62577497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:48.239846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-086fba6a-a612-4cd0-9098-31ef62577497 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:48.240926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-086fba6a-a612-4cd0-9098-31ef62577497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:48.282991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:48.349599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 701/1393] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:48.469368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ba28-583d-47a1-b450-5d0d5c199453 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:48.469771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ba28-583d-47a1-b450-5d0d5c199453 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:48.470045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ba28-583d-47a1-b450-5d0d5c199453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:48.536906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0fd0ba28-583d-47a1-b450-5d0d5c199453 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:48.538141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0fd0ba28-583d-47a1-b450-5d0d5c199453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:48.592327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:48.623945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 693/1394] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:41:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:48.640403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fba6a-a612-4cd0-9098-31ef62577497 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:48.642159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-086fba6a-a612-4cd0-9098-31ef62577497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:48.670747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2f737-710a-46e5-9f6e-d648e628da6f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:48.671107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2f737-710a-46e5-9f6e-d648e628da6f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:48.671450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2f737-710a-46e5-9f6e-d648e628da6f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:48.672938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 694/1395] 10.222.0.15 () {54 vars in 943 bytes} [Fri Sep 9 01:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 01:41:48.679298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919298-2ce9-4c19-9123-ca3c3edb23f6 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:48.679577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919298-2ce9-4c19-9123-ca3c3edb23f6 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:48.679846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919298-2ce9-4c19-9123-ca3c3edb23f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:48.755977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fba6a-a612-4cd0-9098-31ef62577497 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:48.756993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-aa919298-2ce9-4c19-9123-ca3c3edb23f6 None None] MFA Rules not processed for user `e0d08965bc9c4ca29812d304bfb4ab4f`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:48.758295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-aa919298-2ce9-4c19-9123-ca3c3edb23f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:48.811668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:48.864642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 695/1396] 10.222.0.15 () {58 vars in 1040 bytes} [Fri Sep 9 01:41:48 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:41:48.871558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2a1ee-3246-4cff-a224-baa1d89cbd42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:48.872195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a1ee-3246-4cff-a224-baa1d89cbd42 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:48.872464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a1ee-3246-4cff-a224-baa1d89cbd42 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:48.872729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2a1ee-3246-4cff-a224-baa1d89cbd42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:48.874181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa919298-2ce9-4c19-9123-ca3c3edb23f6 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:48.874335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f7b2a1ee-3246-4cff-a224-baa1d89cbd42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:48.884462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa919298-2ce9-4c19-9123-ca3c3edb23f6 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:48.910156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:48.970703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a1ee-3246-4cff-a224-baa1d89cbd42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:48.972302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2a1ee-3246-4cff-a224-baa1d89cbd42 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:48.988275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b801dd-8041-4489-9664-82de68ad0d26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:48.988873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b801dd-8041-4489-9664-82de68ad0d26 service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:48.989178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b801dd-8041-4489-9664-82de68ad0d26 service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:48.989448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b801dd-8041-4489-9664-82de68ad0d26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:48.990583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 702/1397] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:48.991051 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-19b801dd-8041-4489-9664-82de68ad0d26 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:49.078566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b801dd-8041-4489-9664-82de68ad0d26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:49.080180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b801dd-8041-4489-9664-82de68ad0d26 service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:49.090204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 696/1398] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:41:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 01:41:51.272656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467fd55-d190-436d-a8be-a93a9d142207 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:51.273654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467fd55-d190-436d-a8be-a93a9d142207 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:51.274392 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467fd55-d190-436d-a8be-a93a9d142207 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:51.275924 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 703/1399] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:51.281307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663277ea-f1d1-4e98-b110-5b44007ca0bf None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:51.282140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663277ea-f1d1-4e98-b110-5b44007ca0bf None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:51.282421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663277ea-f1d1-4e98-b110-5b44007ca0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:51.363126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-663277ea-f1d1-4e98-b110-5b44007ca0bf None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:51.364432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-663277ea-f1d1-4e98-b110-5b44007ca0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:51.371550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:51.477150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 697/1400] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:51.780407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467fd55-d190-436d-a8be-a93a9d142207 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:51.781971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9467fd55-d190-436d-a8be-a93a9d142207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:51.854763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:51.954645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467fd55-d190-436d-a8be-a93a9d142207 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:52.105370 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc8ce0-b7e7-4343-8e5f-b552ddd924ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:52.106092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8ce0-b7e7-4343-8e5f-b552ddd924ec service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:52.106395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8ce0-b7e7-4343-8e5f-b552ddd924ec service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:52.106670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8ce0-b7e7-4343-8e5f-b552ddd924ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:52.108383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d0dc8ce0-b7e7-4343-8e5f-b552ddd924ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:52.140437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:52.214992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc8ce0-b7e7-4343-8e5f-b552ddd924ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:52.216703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc8ce0-b7e7-4343-8e5f-b552ddd924ec service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:52.236815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 704/1401] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:54.511590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7a9a-8554-4820-8c8f-370e69fcfe34 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:54.521649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7a9a-8554-4820-8c8f-370e69fcfe34 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:54.521649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba7a9a-8554-4820-8c8f-370e69fcfe34 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:54.521649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 698/1402] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:54.524921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f21c2-d3bb-42cc-b01c-d4bb7d2f6373 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:54.525394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f21c2-d3bb-42cc-b01c-d4bb7d2f6373 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:54.525498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f21c2-d3bb-42cc-b01c-d4bb7d2f6373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:54.593320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-532f21c2-d3bb-42cc-b01c-d4bb7d2f6373 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:54.594533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-532f21c2-d3bb-42cc-b01c-d4bb7d2f6373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:54.707483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 705/1403] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:55.001485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7a9a-8554-4820-8c8f-370e69fcfe34 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:55.003244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5eba7a9a-8554-4820-8c8f-370e69fcfe34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:55.153909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:55.164527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba7a9a-8554-4820-8c8f-370e69fcfe34 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:55.295218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5bdf-b0e7-4b64-b1d3-c610b308e6f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:55.295822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5bdf-b0e7-4b64-b1d3-c610b308e6f4 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:55.296207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5bdf-b0e7-4b64-b1d3-c610b308e6f4 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:55.296480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5bdf-b0e7-4b64-b1d3-c610b308e6f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:55.298160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-234a5bdf-b0e7-4b64-b1d3-c610b308e6f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:55.395872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a5bdf-b0e7-4b64-b1d3-c610b308e6f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:55.397638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a5bdf-b0e7-4b64-b1d3-c610b308e6f4 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:55.408913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:55.418690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 699/1404] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:41:57.660438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32da7-3b1e-400e-873c-3b53e8d9b8bd None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:57.661024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32da7-3b1e-400e-873c-3b53e8d9b8bd None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:57.661024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a32da7-3b1e-400e-873c-3b53e8d9b8bd None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:57.665101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 706/1405] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:41:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:41:57.668359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa02cc-16fd-418a-84f6-e2a4ca2f4e71 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:57.668785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa02cc-16fd-418a-84f6-e2a4ca2f4e71 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:57.669075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa02cc-16fd-418a-84f6-e2a4ca2f4e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:57.739643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-58fa02cc-16fd-418a-84f6-e2a4ca2f4e71 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:41:57.740812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-58fa02cc-16fd-418a-84f6-e2a4ca2f4e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:57.852988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 700/1406] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:41:58.147404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a32da7-3b1e-400e-873c-3b53e8d9b8bd None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:41:58.149537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b4a32da7-3b1e-400e-873c-3b53e8d9b8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:58.154218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:58.320642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a32da7-3b1e-400e-873c-3b53e8d9b8bd None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:41:58.404605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:41:58.454594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedee2dc-5626-4f8e-ae2d-81def03ac98b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:41:58.455240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedee2dc-5626-4f8e-ae2d-81def03ac98b service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:41:58.455533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedee2dc-5626-4f8e-ae2d-81def03ac98b service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:41:58.455791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedee2dc-5626-4f8e-ae2d-81def03ac98b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:41:58.457460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-fedee2dc-5626-4f8e-ae2d-81def03ac98b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:41:58.572223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedee2dc-5626-4f8e-ae2d-81def03ac98b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:41:58.574352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedee2dc-5626-4f8e-ae2d-81def03ac98b service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:41:58.595912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 707/1407] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:07.851493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=67 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:08.593497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=61 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:12.125042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e821-80e3-4d04-a22b-ded634e7e053 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:12.125608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e821-80e3-4d04-a22b-ded634e7e053 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:12.125752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e821-80e3-4d04-a22b-ded634e7e053 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:12.127136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 701/1408] 10.222.0.15 () {54 vars in 918 bytes} [Fri Sep 9 01:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:12.132878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd966d-4515-43af-a3e3-b300ea1c9195 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:12.133308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd966d-4515-43af-a3e3-b300ea1c9195 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:12.133447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd966d-4515-43af-a3e3-b300ea1c9195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:12.207291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6bdd966d-4515-43af-a3e3-b300ea1c9195 None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:12.208424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6bdd966d-4515-43af-a3e3-b300ea1c9195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:12.313838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 708/1409] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 9 01:42:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:12.327690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eabde-4f71-4337-9498-2a7cc4b29edd None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:12.327976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eabde-4f71-4337-9498-2a7cc4b29edd None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:12.328250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eabde-4f71-4337-9498-2a7cc4b29edd None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:12.329682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 702/1410] 10.222.0.15 () {54 vars in 918 bytes} [Fri Sep 9 01:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:12.334077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3677f26-9f2f-4904-92a4-6ea86c364212 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:12.334360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3677f26-9f2f-4904-92a4-6ea86c364212 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:12.334630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3677f26-9f2f-4904-92a4-6ea86c364212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:12.405945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:12.408891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b3677f26-9f2f-4904-92a4-6ea86c364212 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:12.410747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b3677f26-9f2f-4904-92a4-6ea86c364212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:12.524021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 709/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 9 01:42:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:15.075461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cffc05-3e16-4095-a822-2b01af0eb60d None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:15.076023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cffc05-3e16-4095-a822-2b01af0eb60d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:15.076023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cffc05-3e16-4095-a822-2b01af0eb60d None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:15.077426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 703/1412] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:15.082923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfe78-22ce-44e5-b03e-244eefa11e17 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:15.083346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfe78-22ce-44e5-b03e-244eefa11e17 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:15.083472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cfe78-22ce-44e5-b03e-244eefa11e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:15.149110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-441cfe78-22ce-44e5-b03e-244eefa11e17 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:15.150255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-441cfe78-22ce-44e5-b03e-244eefa11e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:15.204692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:15.246396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 710/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:15.539131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cffc05-3e16-4095-a822-2b01af0eb60d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:15.540637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-72cffc05-3e16-4095-a822-2b01af0eb60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:15.683944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cffc05-3e16-4095-a822-2b01af0eb60d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:15.754450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:15.794548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b1500-6337-44bb-80ef-c258abdcec29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:15.795147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b1500-6337-44bb-80ef-c258abdcec29 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:15.795424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b1500-6337-44bb-80ef-c258abdcec29 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:15.795696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b1500-6337-44bb-80ef-c258abdcec29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:15.797933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-794b1500-6337-44bb-80ef-c258abdcec29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:15.894621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b1500-6337-44bb-80ef-c258abdcec29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:15.896228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b1500-6337-44bb-80ef-c258abdcec29 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:15.914420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 704/1414] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:17.060697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e962fe8-f72e-4d87-9358-bf186c4364be None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:17.061273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e962fe8-f72e-4d87-9358-bf186c4364be None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:17.061273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e962fe8-f72e-4d87-9358-bf186c4364be None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:17.062628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 711/1415] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:17.068833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf89b-3078-4459-96c9-1d6329b0b77e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:17.069280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf89b-3078-4459-96c9-1d6329b0b77e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:17.069998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cf89b-3078-4459-96c9-1d6329b0b77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:17.144333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f72cf89b-3078-4459-96c9-1d6329b0b77e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:17.145568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f72cf89b-3078-4459-96c9-1d6329b0b77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:17.172421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:17.241678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 705/1416] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:17.393149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e962fe8-f72e-4d87-9358-bf186c4364be None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:17.394641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4e962fe8-f72e-4d87-9358-bf186c4364be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:17.539108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e962fe8-f72e-4d87-9358-bf186c4364be None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:17.578322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:17.650791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2667b04-7c25-497c-8e7e-0504d860e788 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:17.651382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2667b04-7c25-497c-8e7e-0504d860e788 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:17.651656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2667b04-7c25-497c-8e7e-0504d860e788 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:17.651925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2667b04-7c25-497c-8e7e-0504d860e788 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:17.653594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c2667b04-7c25-497c-8e7e-0504d860e788 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:17.740451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2667b04-7c25-497c-8e7e-0504d860e788 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:17.742518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2667b04-7c25-497c-8e7e-0504d860e788 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:17.760975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 712/1417] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:18.880817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd690-5c43-4ec6-abe8-4ea5affd51f5 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:18.881455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd690-5c43-4ec6-abe8-4ea5affd51f5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:18.881455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd690-5c43-4ec6-abe8-4ea5affd51f5 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:18.882786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 706/1418] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:18.889195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c5d1f-7ab0-4e1e-a763-64c45bc6da82 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:18.889644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c5d1f-7ab0-4e1e-a763-64c45bc6da82 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:18.889737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c5d1f-7ab0-4e1e-a763-64c45bc6da82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:18.944391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:18.965779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ea4c5d1f-7ab0-4e1e-a763-64c45bc6da82 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:18.966963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ea4c5d1f-7ab0-4e1e-a763-64c45bc6da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:19.062168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 713/1419] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:19.217105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cd690-5c43-4ec6-abe8-4ea5affd51f5 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:19.218527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-dd7cd690-5c43-4ec6-abe8-4ea5affd51f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:19.364803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cd690-5c43-4ec6-abe8-4ea5affd51f5 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:19.381309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:19.481117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a666e-217a-402c-85df-09768791a0d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:19.481748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a666e-217a-402c-85df-09768791a0d3 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:19.482022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a666e-217a-402c-85df-09768791a0d3 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:19.482290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a666e-217a-402c-85df-09768791a0d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:19.483853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-835a666e-217a-402c-85df-09768791a0d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:19.573496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a666e-217a-402c-85df-09768791a0d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:19.575080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a666e-217a-402c-85df-09768791a0d3 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:19.593167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 707/1420] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:20.702837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4c5f3-75dd-4a55-811d-58fedcb2ba20 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:20.703379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4c5f3-75dd-4a55-811d-58fedcb2ba20 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:20.703379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4c5f3-75dd-4a55-811d-58fedcb2ba20 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:20.704769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 714/1421] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:20.711631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d783e4-8aa9-4f94-9b7b-7153bb53b204 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:20.712052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d783e4-8aa9-4f94-9b7b-7153bb53b204 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:20.712208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d783e4-8aa9-4f94-9b7b-7153bb53b204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:20.732141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:20.790311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-37d783e4-8aa9-4f94-9b7b-7153bb53b204 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:20.791511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-37d783e4-8aa9-4f94-9b7b-7153bb53b204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:20.889336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 708/1422] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:21.043353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4c5f3-75dd-4a55-811d-58fedcb2ba20 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:21.045300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2ac4c5f3-75dd-4a55-811d-58fedcb2ba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:21.152594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:21.187645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4c5f3-75dd-4a55-811d-58fedcb2ba20 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:21.299568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781c505-4a39-4250-9fd6-69a35f5ec4a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:21.300100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781c505-4a39-4250-9fd6-69a35f5ec4a4 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:21.300377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781c505-4a39-4250-9fd6-69a35f5ec4a4 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:21.300650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781c505-4a39-4250-9fd6-69a35f5ec4a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:21.302303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c781c505-4a39-4250-9fd6-69a35f5ec4a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:21.376113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:21.391008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781c505-4a39-4250-9fd6-69a35f5ec4a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:21.392710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781c505-4a39-4250-9fd6-69a35f5ec4a4 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:21.410963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 715/1423] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:22.526926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3422ea-0604-48ec-aaa4-0e2e192eac6f None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:22.527381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3422ea-0604-48ec-aaa4-0e2e192eac6f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:22.527475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3422ea-0604-48ec-aaa4-0e2e192eac6f None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:22.528832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 709/1424] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:22.535300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de274a-8e24-4da7-a445-578ac960ec3c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:22.535739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de274a-8e24-4da7-a445-578ac960ec3c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:22.535880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de274a-8e24-4da7-a445-578ac960ec3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:22.612426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-80de274a-8e24-4da7-a445-578ac960ec3c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:22.613652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-80de274a-8e24-4da7-a445-578ac960ec3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:22.709270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 716/1425] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:22.859083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3422ea-0604-48ec-aaa4-0e2e192eac6f None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:22.860597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bc3422ea-0604-48ec-aaa4-0e2e192eac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:22.943695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:23.004948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3422ea-0604-48ec-aaa4-0e2e192eac6f None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:23.116346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda61b2-3beb-408d-af57-c472dece0585 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:23.116904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda61b2-3beb-408d-af57-c472dece0585 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:23.117210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda61b2-3beb-408d-af57-c472dece0585 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:23.117479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda61b2-3beb-408d-af57-c472dece0585 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:23.119050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-eeda61b2-3beb-408d-af57-c472dece0585 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:23.167864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:23.208307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda61b2-3beb-408d-af57-c472dece0585 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:23.209921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda61b2-3beb-408d-af57-c472dece0585 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:23.227682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 710/1426] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:24.276257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d53ca-c484-4892-a471-95c735cc64c8 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:24.276801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d53ca-c484-4892-a471-95c735cc64c8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:24.276801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d53ca-c484-4892-a471-95c735cc64c8 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:24.278129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 717/1427] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:24.284347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36bc3f-a572-4e30-9923-8481279a8773 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:24.284778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36bc3f-a572-4e30-9923-8481279a8773 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:24.284884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36bc3f-a572-4e30-9923-8481279a8773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:24.361448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7d36bc3f-a572-4e30-9923-8481279a8773 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:24.362643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7d36bc3f-a572-4e30-9923-8481279a8773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:24.458151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 711/1428] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:24.611240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d53ca-c484-4892-a471-95c735cc64c8 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:24.612795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0e7d53ca-c484-4892-a471-95c735cc64c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:24.660915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:24.756023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d53ca-c484-4892-a471-95c735cc64c8 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:24.866364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e9042-c9eb-4b68-99d0-ddaf4b9f5ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:24.866882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e9042-c9eb-4b68-99d0-ddaf4b9f5ec5 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:24.867153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e9042-c9eb-4b68-99d0-ddaf4b9f5ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:24.867419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e9042-c9eb-4b68-99d0-ddaf4b9f5ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:24.868988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-fa8e9042-c9eb-4b68-99d0-ddaf4b9f5ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:24.875762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:24.958441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e9042-c9eb-4b68-99d0-ddaf4b9f5ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:24.960637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e9042-c9eb-4b68-99d0-ddaf4b9f5ec5 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:24.978298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 718/1429] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:26.038691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36efecb-55aa-499e-a4a5-f54efdd096a0 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:26.039221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36efecb-55aa-499e-a4a5-f54efdd096a0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:26.039221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36efecb-55aa-499e-a4a5-f54efdd096a0 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:26.040568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 712/1430] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:26.047174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a78a5-08fa-4616-b69c-6ee32c3978bf None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:26.047174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a78a5-08fa-4616-b69c-6ee32c3978bf None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:26.047611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a78a5-08fa-4616-b69c-6ee32c3978bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:26.122011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5f1a78a5-08fa-4616-b69c-6ee32c3978bf None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:26.123219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5f1a78a5-08fa-4616-b69c-6ee32c3978bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:26.190435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:26.218724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 719/1431] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:26.372109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36efecb-55aa-499e-a4a5-f54efdd096a0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:26.373682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e36efecb-55aa-499e-a4a5-f54efdd096a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:26.391133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:26.516833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36efecb-55aa-499e-a4a5-f54efdd096a0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:26.606402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:26.633052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e9bcd-8dc4-4ad9-8ec6-10455ad27ad2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:26.633623 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e9bcd-8dc4-4ad9-8ec6-10455ad27ad2 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:26.633900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e9bcd-8dc4-4ad9-8ec6-10455ad27ad2 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:26.634170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e9bcd-8dc4-4ad9-8ec6-10455ad27ad2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:26.635752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-912e9bcd-8dc4-4ad9-8ec6-10455ad27ad2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:26.726748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e9bcd-8dc4-4ad9-8ec6-10455ad27ad2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:26.728335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e9bcd-8dc4-4ad9-8ec6-10455ad27ad2 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:26.746523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 713/1432] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:27.796368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548c4a-648b-4076-907b-673f47e662f9 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:27.796858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548c4a-648b-4076-907b-673f47e662f9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:27.796858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c548c4a-648b-4076-907b-673f47e662f9 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:27.798490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 720/1433] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:27.804778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4025c2b-9eab-4ed3-9f7a-82804ae6b1f7 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:27.805257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4025c2b-9eab-4ed3-9f7a-82804ae6b1f7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:27.805352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4025c2b-9eab-4ed3-9f7a-82804ae6b1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:27.880879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d4025c2b-9eab-4ed3-9f7a-82804ae6b1f7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:27.882637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d4025c2b-9eab-4ed3-9f7a-82804ae6b1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:27.924366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:27.980592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 714/1434] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:28.131992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c548c4a-648b-4076-907b-673f47e662f9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:28.133509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5c548c4a-648b-4076-907b-673f47e662f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:28.278683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c548c4a-648b-4076-907b-673f47e662f9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:28.331958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:28.389970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d8162-2644-4cac-96f9-53b3ae4797b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:28.390503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8162-2644-4cac-96f9-53b3ae4797b7 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:28.390782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8162-2644-4cac-96f9-53b3ae4797b7 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:28.391056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8162-2644-4cac-96f9-53b3ae4797b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:28.392641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-972d8162-2644-4cac-96f9-53b3ae4797b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:28.479283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d8162-2644-4cac-96f9-53b3ae4797b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:28.480859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d8162-2644-4cac-96f9-53b3ae4797b7 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:28.498783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 721/1435] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:29.552054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4504b1-6465-4250-bc83-40fbc8d6b473 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:29.552587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4504b1-6465-4250-bc83-40fbc8d6b473 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:29.552587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4504b1-6465-4250-bc83-40fbc8d6b473 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:29.553982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 715/1436] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:29.560097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513dc7a-0d74-43e0-b90a-5ed420b6b751 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:29.560527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513dc7a-0d74-43e0-b90a-5ed420b6b751 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:29.560631 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513dc7a-0d74-43e0-b90a-5ed420b6b751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:29.636287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f513dc7a-0d74-43e0-b90a-5ed420b6b751 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:29.637540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f513dc7a-0d74-43e0-b90a-5ed420b6b751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:29.639515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:29.732049 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 722/1437] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:42:29.883488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4504b1-6465-4250-bc83-40fbc8d6b473 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:29.884978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ed4504b1-6465-4250-bc83-40fbc8d6b473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:30.028406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4504b1-6465-4250-bc83-40fbc8d6b473 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:30.055968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:30.141052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08b8d3-54d8-4f1a-9441-786887c121ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:30.141677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08b8d3-54d8-4f1a-9441-786887c121ba service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:30.141951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08b8d3-54d8-4f1a-9441-786887c121ba service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:30.142219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08b8d3-54d8-4f1a-9441-786887c121ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:30.143801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fc08b8d3-54d8-4f1a-9441-786887c121ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:30.231427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08b8d3-54d8-4f1a-9441-786887c121ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:30.233053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08b8d3-54d8-4f1a-9441-786887c121ba service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:30.250872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 716/1438] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:32.328562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfccb7-0100-465f-8197-c9ec7de4e787 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:32.329126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfccb7-0100-465f-8197-c9ec7de4e787 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:32.329126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfccb7-0100-465f-8197-c9ec7de4e787 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:32.330454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 723/1439] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:32.336506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebbd5-6d20-4fe8-a708-f053579fa39e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:32.336930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebbd5-6d20-4fe8-a708-f053579fa39e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:32.337023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ebbd5-6d20-4fe8-a708-f053579fa39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:32.376444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:32.403273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-442ebbd5-6d20-4fe8-a708-f053579fa39e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:32.404448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-442ebbd5-6d20-4fe8-a708-f053579fa39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:32.497163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 717/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:32.784930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfccb7-0100-465f-8197-c9ec7de4e787 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:32.786476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-72bfccb7-0100-465f-8197-c9ec7de4e787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:32.888722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfccb7-0100-465f-8197-c9ec7de4e787 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:32.921298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:32.962883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3ab65-41c9-441a-b792-c6552218dba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:32.963487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ab65-41c9-441a-b792-c6552218dba8 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:32.963761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ab65-41c9-441a-b792-c6552218dba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:32.964027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ab65-41c9-441a-b792-c6552218dba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:32.965612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-dda3ab65-41c9-441a-b792-c6552218dba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:33.070824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3ab65-41c9-441a-b792-c6552218dba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:33.072296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3ab65-41c9-441a-b792-c6552218dba8 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:33.089852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 724/1441] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:35.373922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5869-5a16-4f9c-a755-a8cf8fd290c5 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:35.374456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5869-5a16-4f9c-a755-a8cf8fd290c5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:35.374553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c5869-5a16-4f9c-a755-a8cf8fd290c5 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:35.375910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 718/1442] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:35.381946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3abbff-3e32-4bc2-8f89-d7b92ac29149 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:35.382343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3abbff-3e32-4bc2-8f89-d7b92ac29149 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:35.382500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3abbff-3e32-4bc2-8f89-d7b92ac29149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:35.446997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-bc3abbff-3e32-4bc2-8f89-d7b92ac29149 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:35.448200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-bc3abbff-3e32-4bc2-8f89-d7b92ac29149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:35.461252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:35.545473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 725/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:35.826926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c5869-5a16-4f9c-a755-a8cf8fd290c5 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:35.828494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c78c5869-5a16-4f9c-a755-a8cf8fd290c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:35.931192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c5869-5a16-4f9c-a755-a8cf8fd290c5 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:35.948081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:36.010993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fa618-a97e-4c01-934b-05a66cd0012e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:36.011589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fa618-a97e-4c01-934b-05a66cd0012e service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:36.011862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fa618-a97e-4c01-934b-05a66cd0012e service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:36.012129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fa618-a97e-4c01-934b-05a66cd0012e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:36.013724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f40fa618-a97e-4c01-934b-05a66cd0012e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:36.118424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fa618-a97e-4c01-934b-05a66cd0012e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:36.119861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fa618-a97e-4c01-934b-05a66cd0012e service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:36.138417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 719/1444] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:38.585219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e292b-7016-4d21-aa77-e048012725a2 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:38.585758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e292b-7016-4d21-aa77-e048012725a2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:38.585758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e292b-7016-4d21-aa77-e048012725a2 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:38.587097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 726/1445] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:38.592410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be830b-3cc8-44b8-9dc5-c740595dc822 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:38.592790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be830b-3cc8-44b8-9dc5-c740595dc822 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:38.592958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be830b-3cc8-44b8-9dc5-c740595dc822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:38.647754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:38.667565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-95be830b-3cc8-44b8-9dc5-c740595dc822 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:38.668775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-95be830b-3cc8-44b8-9dc5-c740595dc822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:38.762967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 720/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:39.048332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e292b-7016-4d21-aa77-e048012725a2 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:39.049913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d73e292b-7016-4d21-aa77-e048012725a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:39.150579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e292b-7016-4d21-aa77-e048012725a2 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:39.190437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:39.227216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941ae2f-5aac-4228-8f44-a6c1db26792a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:39.227825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941ae2f-5aac-4228-8f44-a6c1db26792a service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:39.228101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941ae2f-5aac-4228-8f44-a6c1db26792a service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:39.228369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941ae2f-5aac-4228-8f44-a6c1db26792a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:39.230007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3941ae2f-5aac-4228-8f44-a6c1db26792a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:39.338464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941ae2f-5aac-4228-8f44-a6c1db26792a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:39.339896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941ae2f-5aac-4228-8f44-a6c1db26792a service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:39.358045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 727/1447] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:41.634706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b8343-3e37-4056-a3a0-f2ed08848959 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:41.635592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b8343-3e37-4056-a3a0-f2ed08848959 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:41.635868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b8343-3e37-4056-a3a0-f2ed08848959 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:41.637277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 721/1448] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:41.642586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece7ca-442c-4345-8708-b465d15f16e8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:41.642990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece7ca-442c-4345-8708-b465d15f16e8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:41.643161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece7ca-442c-4345-8708-b465d15f16e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:41.709671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5cece7ca-442c-4345-8708-b465d15f16e8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:41.710912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5cece7ca-442c-4345-8708-b465d15f16e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:41.731267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:41.807716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 728/1449] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:42.091834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b8343-3e37-4056-a3a0-f2ed08848959 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:42.093372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-691b8343-3e37-4056-a3a0-f2ed08848959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:42.193707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b8343-3e37-4056-a3a0-f2ed08848959 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:42.213766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:42.269363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bf1ee-1d14-42ee-8102-58f3bdcd4bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:42.269969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf1ee-1d14-42ee-8102-58f3bdcd4bbd service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:42.270248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf1ee-1d14-42ee-8102-58f3bdcd4bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:42.270518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bf1ee-1d14-42ee-8102-58f3bdcd4bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:42.272275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-976bf1ee-1d14-42ee-8102-58f3bdcd4bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:42.385540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bf1ee-1d14-42ee-8102-58f3bdcd4bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:42.386928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bf1ee-1d14-42ee-8102-58f3bdcd4bbd service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:42.405169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 722/1450] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:44.618968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbfc5b-1752-429d-89bc-78faa3ea5ce4 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:44.619577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbfc5b-1752-429d-89bc-78faa3ea5ce4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:44.619577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbfc5b-1752-429d-89bc-78faa3ea5ce4 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:44.620872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 729/1451] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:44.626142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4553f-156e-46d5-a8a8-5b1715eb407f None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:44.626566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4553f-156e-46d5-a8a8-5b1715eb407f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:44.626685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4553f-156e-46d5-a8a8-5b1715eb407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:44.689008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:44.691704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b4a4553f-156e-46d5-a8a8-5b1715eb407f None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:44.693493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b4a4553f-156e-46d5-a8a8-5b1715eb407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:44.789018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 723/1452] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:45.092885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b6ada-4948-4f7e-8fbb-c0ed75e4493b None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:45.093203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b6ada-4948-4f7e-8fbb-c0ed75e4493b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:45.093479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b6ada-4948-4f7e-8fbb-c0ed75e4493b None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:45.094813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 730/1453] 10.222.0.15 () {54 vars in 965 bytes} [Fri Sep 9 01:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:45.100681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0c98d-4d5d-4ff9-9ed2-0f2046f1dcc3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:45.100975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0c98d-4d5d-4ff9-9ed2-0f2046f1dcc3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:45.101279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0c98d-4d5d-4ff9-9ed2-0f2046f1dcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:45.175713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-fef0c98d-4d5d-4ff9-9ed2-0f2046f1dcc3 None None] MFA Rules not processed for user `341277e7e3144e5ea93fa657916821e1`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:45.176841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fef0c98d-4d5d-4ff9-9ed2-0f2046f1dcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:45.244742 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:45.250887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 724/1454] 10.222.0.15 () {58 vars in 1062 bytes} [Fri Sep 9 01:42:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:42:45.260662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b6ada-4948-4f7e-8fbb-c0ed75e4493b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:45.262204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-334b6ada-4948-4f7e-8fbb-c0ed75e4493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:45.363580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b6ada-4948-4f7e-8fbb-c0ed75e4493b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:45.412170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:45.436419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ae029-a7d6-4f63-8870-06555c4240c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:45.436947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ae029-a7d6-4f63-8870-06555c4240c0 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:45.437254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ae029-a7d6-4f63-8870-06555c4240c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:45.437523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ae029-a7d6-4f63-8870-06555c4240c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:45.439085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9e1ae029-a7d6-4f63-8870-06555c4240c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:45.544390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ae029-a7d6-4f63-8870-06555c4240c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:45.545830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ae029-a7d6-4f63-8870-06555c4240c0 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:45.563954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 731/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:48.240613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786de5e-b3a6-4820-acd1-e6125e3d95c9 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:48.241161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786de5e-b3a6-4820-acd1-e6125e3d95c9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:48.241161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786de5e-b3a6-4820-acd1-e6125e3d95c9 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:48.242451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:48.247764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6f392-bc49-44f8-b1af-f83e490ca65c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:48.248204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6f392-bc49-44f8-b1af-f83e490ca65c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:48.248311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6f392-bc49-44f8-b1af-f83e490ca65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:48.312830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1cb6f392-bc49-44f8-b1af-f83e490ca65c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:48.314091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1cb6f392-bc49-44f8-b1af-f83e490ca65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:48.342899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:48.409885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:48.697043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786de5e-b3a6-4820-acd1-e6125e3d95c9 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:48.698575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9786de5e-b3a6-4820-acd1-e6125e3d95c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:48.800274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786de5e-b3a6-4820-acd1-e6125e3d95c9 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:48.872189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccce80-0133-4fb0-b643-50e9dc6c4a62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:48.872782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccce80-0133-4fb0-b643-50e9dc6c4a62 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:48.873054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccce80-0133-4fb0-b643-50e9dc6c4a62 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:48.873351 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccce80-0133-4fb0-b643-50e9dc6c4a62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:48.874913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-04ccce80-0133-4fb0-b643-50e9dc6c4a62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:48.924282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:48.984770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccce80-0133-4fb0-b643-50e9dc6c4a62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:48.986150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccce80-0133-4fb0-b643-50e9dc6c4a62 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:49.006193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 726/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:51.266019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f707-7d9a-447f-a64f-30ea824464b7 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:51.266560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f707-7d9a-447f-a64f-30ea824464b7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:51.266560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108f707-7d9a-447f-a64f-30ea824464b7 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:51.268194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:51.273802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eddca5-2260-43d5-b03f-51b58f0d6af7 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:51.274242 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eddca5-2260-43d5-b03f-51b58f0d6af7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:51.274346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eddca5-2260-43d5-b03f-51b58f0d6af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:51.339067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-38eddca5-2260-43d5-b03f-51b58f0d6af7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:51.340283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-38eddca5-2260-43d5-b03f-51b58f0d6af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:51.435475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:51.739912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108f707-7d9a-447f-a64f-30ea824464b7 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:51.741569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e108f707-7d9a-447f-a64f-30ea824464b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:51.963353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108f707-7d9a-447f-a64f-30ea824464b7 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:52.018376 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:52.037647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc233a1-96c2-445e-b2ba-16f4c07579b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:52.038251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc233a1-96c2-445e-b2ba-16f4c07579b4 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:52.038527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc233a1-96c2-445e-b2ba-16f4c07579b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:52.038799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc233a1-96c2-445e-b2ba-16f4c07579b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:52.040372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-bdc233a1-96c2-445e-b2ba-16f4c07579b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:52.150711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc233a1-96c2-445e-b2ba-16f4c07579b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:52.152119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc233a1-96c2-445e-b2ba-16f4c07579b4 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:52.169618 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 734/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:42:54.378693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5c02-ecc3-4445-a5a5-665caac9ff82 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:54.379243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5c02-ecc3-4445-a5a5-665caac9ff82 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:54.379243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5c02-ecc3-4445-a5a5-665caac9ff82 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:54.381137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:54.386680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4e587-89ae-4ae4-9069-309f781003b2 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:54.387109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4e587-89ae-4ae4-9069-309f781003b2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:54.387250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4e587-89ae-4ae4-9069-309f781003b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:54.452512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b3e4e587-89ae-4ae4-9069-309f781003b2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:54.453750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b3e4e587-89ae-4ae4-9069-309f781003b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:54.491977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:54.550021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:54.838471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e5c02-ecc3-4445-a5a5-665caac9ff82 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:54.840032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-222e5c02-ecc3-4445-a5a5-665caac9ff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:54.843892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:54.943606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e5c02-ecc3-4445-a5a5-665caac9ff82 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:55.015624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124d282-b812-48e2-bdbc-9907146f6630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:55.016215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124d282-b812-48e2-bdbc-9907146f6630 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:55.016487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124d282-b812-48e2-bdbc-9907146f6630 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:55.016754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124d282-b812-48e2-bdbc-9907146f6630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:55.018341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0124d282-b812-48e2-bdbc-9907146f6630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:55.075403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:55.125225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124d282-b812-48e2-bdbc-9907146f6630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:55.126596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124d282-b812-48e2-bdbc-9907146f6630 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:55.145178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 729/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:42:57.306129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bd10e-8760-48dc-9538-86c86ba0b26d None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:57.306637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bd10e-8760-48dc-9538-86c86ba0b26d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:57.306637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524bd10e-8760-48dc-9538-86c86ba0b26d None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:57.307955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:42:57.313304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f30a71-8a5c-4205-be82-a5165f583a41 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:57.313741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f30a71-8a5c-4205-be82-a5165f583a41 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:57.313837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f30a71-8a5c-4205-be82-a5165f583a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:57.380401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-66f30a71-8a5c-4205-be82-a5165f583a41 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:42:57.381617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-66f30a71-8a5c-4205-be82-a5165f583a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:57.474866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:42:57.759858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bd10e-8760-48dc-9538-86c86ba0b26d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:42:57.761439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-524bd10e-8760-48dc-9538-86c86ba0b26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:57.865765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524bd10e-8760-48dc-9538-86c86ba0b26d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:42:57.933447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:42:57.941862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c494b4-7301-4cfb-bd6f-6f53412e9822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:42:57.942996 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c494b4-7301-4cfb-bd6f-6f53412e9822 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:42:57.943265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c494b4-7301-4cfb-bd6f-6f53412e9822 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:42:57.943529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c494b4-7301-4cfb-bd6f-6f53412e9822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:42:57.945106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-00c494b4-7301-4cfb-bd6f-6f53412e9822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:42:58.054852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c494b4-7301-4cfb-bd6f-6f53412e9822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:42:58.056188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c494b4-7301-4cfb-bd6f-6f53412e9822 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:42:58.074850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 737/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:43:00.238491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1591-2b4b-47bf-aa27-f9d186584b7e None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:00.239019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1591-2b4b-47bf-aa27-f9d186584b7e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:00.239019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1591-2b4b-47bf-aa27-f9d186584b7e None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:00.240368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:00.245683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cead6c-0f66-45cd-8962-d853e47f1d26 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:00.246135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cead6c-0f66-45cd-8962-d853e47f1d26 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:00.246241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cead6c-0f66-45cd-8962-d853e47f1d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:00.312084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-96cead6c-0f66-45cd-8962-d853e47f1d26 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:00.313301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-96cead6c-0f66-45cd-8962-d853e47f1d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:00.359425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:00.407523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:43:00.688236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc1591-2b4b-47bf-aa27-f9d186584b7e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:43:00.689793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d0fc1591-2b4b-47bf-aa27-f9d186584b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:00.703302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:00.793738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc1591-2b4b-47bf-aa27-f9d186584b7e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:43:00.864848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cee73-ae41-4528-a94e-81cc9ac17ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:43:00.865591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cee73-ae41-4528-a94e-81cc9ac17ceb service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:00.865863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cee73-ae41-4528-a94e-81cc9ac17ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:00.866123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cee73-ae41-4528-a94e-81cc9ac17ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:00.867677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-668cee73-ae41-4528-a94e-81cc9ac17ceb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:00.934826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:00.976368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cee73-ae41-4528-a94e-81cc9ac17ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:43:00.977874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668cee73-ae41-4528-a94e-81cc9ac17ceb service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:43:00.995469 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 732/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:43:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:43:10.406078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=22 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:10.993174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=25 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:21.022713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39db9bf-93da-4027-9065-fe4c22fbd600 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:21.023300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39db9bf-93da-4027-9065-fe4c22fbd600 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:21.023300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39db9bf-93da-4027-9065-fe4c22fbd600 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:21.024596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 739/1471] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:21.029998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1dcc3-c727-4a12-bd50-5cc9ea96d8e8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:21.030448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1dcc3-c727-4a12-bd50-5cc9ea96d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:21.030566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1dcc3-c727-4a12-bd50-5cc9ea96d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:21.096947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-55c1dcc3-c727-4a12-bd50-5cc9ea96d8e8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:21.098173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-55c1dcc3-c727-4a12-bd50-5cc9ea96d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:21.192628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 733/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:43:21.494221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb86e2-f082-40da-85bf-23a203586c3d None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:21.494501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb86e2-f082-40da-85bf-23a203586c3d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:21.494771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb86e2-f082-40da-85bf-23a203586c3d None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:21.496078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 740/1473] 10.222.0.15 () {54 vars in 966 bytes} [Fri Sep 9 01:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:21.500902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ef55-91aa-4384-9452-7c9f6406d41c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:21.501209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ef55-91aa-4384-9452-7c9f6406d41c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:21.501481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7ef55-91aa-4384-9452-7c9f6406d41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:21.575491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-03a7ef55-91aa-4384-9452-7c9f6406d41c None None] MFA Rules not processed for user `029447e782484e8f8a704b683ede1819`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:21.576667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-03a7ef55-91aa-4384-9452-7c9f6406d41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:21.596490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:21.651949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 734/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Fri Sep 9 01:43:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:43:21.659832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb86e2-f082-40da-85bf-23a203586c3d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:43:21.661345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2bfb86e2-f082-40da-85bf-23a203586c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:21.762235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb86e2-f082-40da-85bf-23a203586c3d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:43:21.834046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9ccce-e5d4-4f52-81f7-a55e389c0ae3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029447e782484e8f8a704b683ede1819', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:43:21.834640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9ccce-e5d4-4f52-81f7-a55e389c0ae3 service glance] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:21.834911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9ccce-e5d4-4f52-81f7-a55e389c0ae3 service glance] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:21.835175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9ccce-e5d4-4f52-81f7-a55e389c0ae3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:21.836725 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e1c9ccce-e5d4-4f52-81f7-a55e389c0ae3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:21.889571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:21.943736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9ccce-e5d4-4f52-81f7-a55e389c0ae3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:43:21.945180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9ccce-e5d4-4f52-81f7-a55e389c0ae3 service glance] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:43:21.963514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 741/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:43:24.216918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17217-3c6f-46f4-b112-118a454a1ffc None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:24.217494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17217-3c6f-46f4-b112-118a454a1ffc None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:24.217494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf17217-3c6f-46f4-b112-118a454a1ffc None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:24.218757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 735/1476] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:24.224448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1f02-0ed0-40ac-9da7-a5325cb25f08 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:24.224851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1f02-0ed0-40ac-9da7-a5325cb25f08 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:24.224979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a1f02-0ed0-40ac-9da7-a5325cb25f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:24.289662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-267a1f02-0ed0-40ac-9da7-a5325cb25f08 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:24.290854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-267a1f02-0ed0-40ac-9da7-a5325cb25f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:24.387874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 742/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:43:24.656286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17217-3c6f-46f4-b112-118a454a1ffc None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:43:24.657875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fdf17217-3c6f-46f4-b112-118a454a1ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:24.781863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf17217-3c6f-46f4-b112-118a454a1ffc None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:43:24.830849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:24.874583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6846f-705a-4ae0-b93e-ef9dce10cf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:43:24.875182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6846f-705a-4ae0-b93e-ef9dce10cf26 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:24.875459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6846f-705a-4ae0-b93e-ef9dce10cf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:24.875729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6846f-705a-4ae0-b93e-ef9dce10cf26 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:24.876365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6846f-705a-4ae0-b93e-ef9dce10cf26 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:43:24.878432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING py.warnings [None req-4bd6846f-705a-4ae0-b93e-ef9dce10cf26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:43:24.878432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: warnings.warn(msg) Sep 09 01:43:24.878432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: Sep 09 01:43:24.880073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6846f-705a-4ae0-b93e-ef9dce10cf26 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:43:24.901186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 736/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Fri Sep 9 01:43:24 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:43:27.123318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74bce3-699b-4523-b029-1001116b0ba6 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:27.123917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74bce3-699b-4523-b029-1001116b0ba6 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:27.123917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74bce3-699b-4523-b029-1001116b0ba6 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:27.125267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 743/1479] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:27.131354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664365f-4c9c-4d11-b6dc-288d71454436 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:27.131824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664365f-4c9c-4d11-b6dc-288d71454436 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:27.132031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664365f-4c9c-4d11-b6dc-288d71454436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:27.202269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-2664365f-4c9c-4d11-b6dc-288d71454436 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:27.203402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2664365f-4c9c-4d11-b6dc-288d71454436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:27.299676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 737/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:43:27.609749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74bce3-699b-4523-b029-1001116b0ba6 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:43:27.611323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6c74bce3-699b-4523-b029-1001116b0ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:27.621605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:27.755068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74bce3-699b-4523-b029-1001116b0ba6 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:43:27.831498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:27.864133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb4bed-7576-47fe-9741-12dcb1d6f345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:43:27.864730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4bed-7576-47fe-9741-12dcb1d6f345 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:27.865004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4bed-7576-47fe-9741-12dcb1d6f345 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:27.865300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4bed-7576-47fe-9741-12dcb1d6f345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:27.866863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-65bb4bed-7576-47fe-9741-12dcb1d6f345 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:27.955395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb4bed-7576-47fe-9741-12dcb1d6f345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:43:27.957041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb4bed-7576-47fe-9741-12dcb1d6f345 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:43:27.975015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 744/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:43:30.301927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3072360-1446-42ed-a3eb-0a5bd3f2a695 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:30.302438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3072360-1446-42ed-a3eb-0a5bd3f2a695 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:30.302438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3072360-1446-42ed-a3eb-0a5bd3f2a695 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:30.303739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 738/1482] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:30.309325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebc7ad-a5cb-46d4-9a96-aabf9d4dddd1 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:30.309715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebc7ad-a5cb-46d4-9a96-aabf9d4dddd1 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:30.309862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebc7ad-a5cb-46d4-9a96-aabf9d4dddd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:30.373289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c9ebc7ad-a5cb-46d4-9a96-aabf9d4dddd1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:30.374467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c9ebc7ad-a5cb-46d4-9a96-aabf9d4dddd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:30.419241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:30.469404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 745/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:43:30.757407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3072360-1446-42ed-a3eb-0a5bd3f2a695 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:43:30.758928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f3072360-1446-42ed-a3eb-0a5bd3f2a695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:30.782594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:30.865309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3072360-1446-42ed-a3eb-0a5bd3f2a695 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:43:30.938627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d337a-f677-4d0a-b7b3-e3d9918f74b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:43:30.939230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d337a-f677-4d0a-b7b3-e3d9918f74b9 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:30.939505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d337a-f677-4d0a-b7b3-e3d9918f74b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:30.939776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d337a-f677-4d0a-b7b3-e3d9918f74b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:30.941382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ff2d337a-f677-4d0a-b7b3-e3d9918f74b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:31.014075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:31.050984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d337a-f677-4d0a-b7b3-e3d9918f74b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:43:31.052443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d337a-f677-4d0a-b7b3-e3d9918f74b9 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:43:31.071442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 739/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:43:33.269379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e488-ee51-4b24-9a2c-1d283d126f62 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:33.269952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e488-ee51-4b24-9a2c-1d283d126f62 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:33.269952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715e488-ee51-4b24-9a2c-1d283d126f62 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:33.271233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 746/1485] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:33.276655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051189b1-2827-47f6-8bea-09b955f06672 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:33.277127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051189b1-2827-47f6-8bea-09b955f06672 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:33.277250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051189b1-2827-47f6-8bea-09b955f06672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:33.462621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-051189b1-2827-47f6-8bea-09b955f06672 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:33.463938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-051189b1-2827-47f6-8bea-09b955f06672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:33.560627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 740/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:43:33.844225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715e488-ee51-4b24-9a2c-1d283d126f62 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:43:33.845812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4715e488-ee51-4b24-9a2c-1d283d126f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:33.951548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715e488-ee51-4b24-9a2c-1d283d126f62 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:43:34.022627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:34.024185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d97c9d-adb8-46d0-adbb-55259df6f236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:43:34.024874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d97c9d-adb8-46d0-adbb-55259df6f236 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:34.025183 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d97c9d-adb8-46d0-adbb-55259df6f236 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:34.025461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d97c9d-adb8-46d0-adbb-55259df6f236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:34.027596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-42d97c9d-adb8-46d0-adbb-55259df6f236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:34.134378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d97c9d-adb8-46d0-adbb-55259df6f236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:43:34.135715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d97c9d-adb8-46d0-adbb-55259df6f236 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:43:34.153656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 747/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:43:36.488131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5bad-ace4-47b6-9a5b-165656da9ec2 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:36.488719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5bad-ace4-47b6-9a5b-165656da9ec2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:36.488719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb5bad-ace4-47b6-9a5b-165656da9ec2 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:36.490071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 741/1488] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:36.495322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63373d2-2ae4-472f-bb1c-6d407cc6dbcd None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:36.495723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63373d2-2ae4-472f-bb1c-6d407cc6dbcd None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:36.495895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63373d2-2ae4-472f-bb1c-6d407cc6dbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:36.560731 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e63373d2-2ae4-472f-bb1c-6d407cc6dbcd None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:36.561971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e63373d2-2ae4-472f-bb1c-6d407cc6dbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:36.613468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:36.660700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 748/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:43:36.953364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb5bad-ace4-47b6-9a5b-165656da9ec2 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:43:36.954883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b0eb5bad-ace4-47b6-9a5b-165656da9ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:36.991197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:37.057036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb5bad-ace4-47b6-9a5b-165656da9ec2 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:43:37.132252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b840ac0-1d7d-4f39-9491-bc2dcdf4d574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:43:37.132850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840ac0-1d7d-4f39-9491-bc2dcdf4d574 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:37.133152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840ac0-1d7d-4f39-9491-bc2dcdf4d574 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:37.133424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b840ac0-1d7d-4f39-9491-bc2dcdf4d574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:37.134994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1b840ac0-1d7d-4f39-9491-bc2dcdf4d574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:37.213676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:37.242546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b840ac0-1d7d-4f39-9491-bc2dcdf4d574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:43:37.243960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b840ac0-1d7d-4f39-9491-bc2dcdf4d574 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:43:37.262090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 742/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:43:39.704004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf6ca-f5cf-4817-816a-5708b66c4a2c None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:39.704565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf6ca-f5cf-4817-816a-5708b66c4a2c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:39.704565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cf6ca-f5cf-4817-816a-5708b66c4a2c None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:39.706449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 749/1491] 10.222.0.15 () {54 vars in 927 bytes} [Fri Sep 9 01:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:43:39.711719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d187a-2c26-4e55-847b-f36f373f5b01 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:39.712140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d187a-2c26-4e55-847b-f36f373f5b01 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:39.712243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d187a-2c26-4e55-847b-f36f373f5b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:39.777680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-497d187a-2c26-4e55-847b-f36f373f5b01 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:43:39.778803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-497d187a-2c26-4e55-847b-f36f373f5b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:39.874189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 743/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Fri Sep 9 01:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:43:40.301630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cf6ca-f5cf-4817-816a-5708b66c4a2c None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:43:40.303375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-cb3cf6ca-f5cf-4817-816a-5708b66c4a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:40.416810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cf6ca-f5cf-4817-816a-5708b66c4a2c None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:43:40.490806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed506be-e86e-4545-a689-79954e8d3261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:43:40.491431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed506be-e86e-4545-a689-79954e8d3261 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:43:40.491706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed506be-e86e-4545-a689-79954e8d3261 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:43:40.491975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed506be-e86e-4545-a689-79954e8d3261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:43:40.493577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7ed506be-e86e-4545-a689-79954e8d3261 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:43:40.502239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:40.599218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed506be-e86e-4545-a689-79954e8d3261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:43:40.600629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed506be-e86e-4545-a689-79954e8d3261 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:43:40.618454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 750/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:43:49.872627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=82 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:43:50.616201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=50 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:54.895554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6cd37-a497-4fbc-a7ef-638faac18085 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:54.896139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6cd37-a497-4fbc-a7ef-638faac18085 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:54.896139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6cd37-a497-4fbc-a7ef-638faac18085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:54.973325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-73b6cd37-a497-4fbc-a7ef-638faac18085 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:44:54.975230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-73b6cd37-a497-4fbc-a7ef-638faac18085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:55.075974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 744/1494] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:44:55.084050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca298-9c0a-44cd-aebe-7039eaf2c792 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:55.084541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca298-9c0a-44cd-aebe-7039eaf2c792 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:55.084645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca298-9c0a-44cd-aebe-7039eaf2c792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:55.159474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5e1ca298-9c0a-44cd-aebe-7039eaf2c792 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:44:55.160597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5e1ca298-9c0a-44cd-aebe-7039eaf2c792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:55.255105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 751/1495] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:44:55.259745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6cd37-a497-4fbc-a7ef-638faac18085 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:55.260934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-73b6cd37-a497-4fbc-a7ef-638faac18085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:55.272082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6cd37-a497-4fbc-a7ef-638faac18085 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:55.337199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:55.370889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:55.371514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:55.371786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:55.372049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:55.372676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:44:55.372972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:55.374586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING py.warnings [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:44:55.374586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: warnings.warn(msg) Sep 09 01:44:55.374586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: Sep 09 01:44:55.375475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:55.386591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 745/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:44:55.394917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca298-9c0a-44cd-aebe-7039eaf2c792 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:55.404467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca298-9c0a-44cd-aebe-7039eaf2c792 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:55.464833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:55.497894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:55.498496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:55.498768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:55.499066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:55.500196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:55.502186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WARNING py.warnings [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:44:55.502186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: warnings.warn(msg) Sep 09 01:44:55.502186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: Sep 09 01:44:55.503371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:55.559619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 752/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:44:55 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:44:55.563759 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:55.565307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:55.597976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57861f74-211e-4819-9891-e68bacad6d99 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:55.688617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:55.689241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:55.689517 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:55.689784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:55.690407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:55.691436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING py.warnings [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:44:55.691436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: warnings.warn(msg) Sep 09 01:44:55.691436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: Sep 09 01:44:55.692654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:55.734064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 746/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:44:55 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:44:55.738251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:55.739741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:55.801762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a23da-63b0-49d1-ace7-d655e9dd1b34 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:55.886413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:55.899025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:55.899638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:55.899916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:55.900185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:55.900823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:55.901925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WARNING py.warnings [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:44:55.901925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: warnings.warn(msg) Sep 09 01:44:55.901925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: Sep 09 01:44:55.902806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:55.915813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 753/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:44:55.919979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:55.921986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:55.933267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2f7f6-a4b3-4846-894e-a34480cd8000 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:55.957713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:56.028622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dfb4a-9c91-4ba4-b3d1-f84fc32d03c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:56.029163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dfb4a-9c91-4ba4-b3d1-f84fc32d03c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:56.029436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dfb4a-9c91-4ba4-b3d1-f84fc32d03c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:56.029695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dfb4a-9c91-4ba4-b3d1-f84fc32d03c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:56.030923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dfb4a-9c91-4ba4-b3d1-f84fc32d03c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:56.031939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING py.warnings [None req-0f7dfb4a-9c91-4ba4-b3d1-f84fc32d03c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:44:56.031939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: warnings.warn(msg) Sep 09 01:44:56.031939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: Sep 09 01:44:56.032810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7dfb4a-9c91-4ba4-b3d1-f84fc32d03c6 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:56.044870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 747/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:44:56.049454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:56.051144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:56.083987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b1294-7918-4148-a66e-fde0067aa2bb admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:56.174562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b1d0c-36d1-4472-b0ab-4c77f356e0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:56.175207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b1d0c-36d1-4472-b0ab-4c77f356e0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:56.175471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b1d0c-36d1-4472-b0ab-4c77f356e0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:56.175740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b1d0c-36d1-4472-b0ab-4c77f356e0b9 admin admin] PATH_INFO: `/v3/projects/f5b96e92eb674b8e84336c0389bc9b3f/users/31f13f43f77e4ef394bddf9096c220a4/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:56.197384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b1d0c-36d1-4472-b0ab-4c77f356e0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b96e92eb674b8e84336c0389bc9b3f, user_id=31f13f43f77e4ef394bddf9096c220a4, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:56.198397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WARNING py.warnings [None req-db7b1d0c-36d1-4472-b0ab-4c77f356e0b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:44:56.198397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: warnings.warn(msg) Sep 09 01:44:56.198397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: Sep 09 01:44:56.200330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b1d0c-36d1-4472-b0ab-4c77f356e0b9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:56.225150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 754/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:44:56 2022] PUT /identity/v3/projects/f5b96e92eb674b8e84336c0389bc9b3f/users/31f13f43f77e4ef394bddf9096c220a4/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:44:56.237694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263ba8-9f8a-4553-9f08-3f3c65efcc75 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:56.237997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263ba8-9f8a-4553-9f08-3f3c65efcc75 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:56.238265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263ba8-9f8a-4553-9f08-3f3c65efcc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:56.295170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f1263ba8-9f8a-4553-9f08-3f3c65efcc75 None None] MFA Rules not processed for user `31f13f43f77e4ef394bddf9096c220a4`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:44:56.296887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f1263ba8-9f8a-4553-9f08-3f3c65efcc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:56.372992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 748/1502] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:44:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:44:56.386354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fba55-d7aa-4d63-b29f-4cd24d0136f0 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:56.386638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fba55-d7aa-4d63-b29f-4cd24d0136f0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:56.386910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fba55-d7aa-4d63-b29f-4cd24d0136f0 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:56.388235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 755/1503] 10.222.0.15 () {54 vars in 964 bytes} [Fri Sep 9 01:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:44:56.393034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e17cd-5c3b-4ae5-8462-7857637a731f None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:56.393365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e17cd-5c3b-4ae5-8462-7857637a731f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:56.393643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e17cd-5c3b-4ae5-8462-7857637a731f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:56.425911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:56.470620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-957e17cd-5c3b-4ae5-8462-7857637a731f None None] MFA Rules not processed for user `df27a2ce80b94237bd2b9ecfe35109cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:44:56.471800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-957e17cd-5c3b-4ae5-8462-7857637a731f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:56.544318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 749/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Fri Sep 9 01:44:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:44:56.552100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fba55-d7aa-4d63-b29f-4cd24d0136f0 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:56.553563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-381fba55-d7aa-4d63-b29f-4cd24d0136f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:56.602571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:56.656696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fba55-d7aa-4d63-b29f-4cd24d0136f0 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:56.728096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:56.728688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:56.728960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:56.729254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:56.730791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:56.827860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:56.829311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:56.832693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:56.848072 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 756/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:44:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:44:56.894289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809011e-8c00-4d31-a71d-17018319577b None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:56.894571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809011e-8c00-4d31-a71d-17018319577b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:56.894840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4809011e-8c00-4d31-a71d-17018319577b None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:56.896132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 750/1506] 10.222.0.15 () {52 vars in 830 bytes} [Fri Sep 9 01:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:44:56.899732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:56.900909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:56.998298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e44ae-8b1b-4b15-b0ef-11213ba514d1 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:57.090458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:57.091022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:57.091302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:57.091573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] PATH_INFO: `/v3/users/31f13f43f77e4ef394bddf9096c220a4` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:57.104895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f13f43f77e4ef394bddf9096c220a4)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:57.105939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WARNING py.warnings [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:44:57.105939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: warnings.warn(msg) Sep 09 01:44:57.105939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: Sep 09 01:44:57.107193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:57.117517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:57.177134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f13f43f77e4ef394bddf9096c220a4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:57.177786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f13f43f77e4ef394bddf9096c220a4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:57.202028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f13f43f77e4ef394bddf9096c220a4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:57.211426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f13f43f77e4ef394bddf9096c220a4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:57.211982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f13f43f77e4ef394bddf9096c220a4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:57.231152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 757/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:44:56 2022] DELETE /identity/v3/users/31f13f43f77e4ef394bddf9096c220a4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:44:57.235493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cb324-6e1c-416a-9535-854df0417776 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:57.235946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cb324-6e1c-416a-9535-854df0417776 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:57.236055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992cb324-6e1c-416a-9535-854df0417776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:57.292385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-992cb324-6e1c-416a-9535-854df0417776 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:44:57.293664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-992cb324-6e1c-416a-9535-854df0417776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:57.370888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:57.383902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 751/1508] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:44:57.394610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:57.395816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:57.493421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826080e-e55f-4794-8ec8-6a720f826a77 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:57.547972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:44:57.569569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8bc129-f069-4ec1-a3a5-9ef15eb35031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:57.570177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc129-f069-4ec1-a3a5-9ef15eb35031 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:57.570448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc129-f069-4ec1-a3a5-9ef15eb35031 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:57.570712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8bc129-f069-4ec1-a3a5-9ef15eb35031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:57.572259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-dd8bc129-f069-4ec1-a3a5-9ef15eb35031 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:57.684690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bc129-f069-4ec1-a3a5-9ef15eb35031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:57.686045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8bc129-f069-4ec1-a3a5-9ef15eb35031 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:57.704217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 758/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:44:58.031546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cb324-6e1c-416a-9535-854df0417776 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:44:58.032853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-992cb324-6e1c-416a-9535-854df0417776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:44:58.044376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992cb324-6e1c-416a-9535-854df0417776 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:44:58.140567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:44:58.141233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:44:58.141510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:44:58.141782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] PATH_INFO: `/v3/projects/f5b96e92eb674b8e84336c0389bc9b3f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:44:58.151651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b96e92eb674b8e84336c0389bc9b3f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:44:58.152701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING py.warnings [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:44:58.152701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: warnings.warn(msg) Sep 09 01:44:58.152701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: Sep 09 01:44:58.153941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:44:58.212181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b96e92eb674b8e84336c0389bc9b3f', 'request_id': 'req-b65d154e-9c9e-425b-8b4d-98e2771c6791'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:58.218150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b96e92eb674b8e84336c0389bc9b3f', 'request_id': 'req-b65d154e-9c9e-425b-8b4d-98e2771c6791'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:58.232122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b96e92eb674b8e84336c0389bc9b3f', 'request_id': 'req-b65d154e-9c9e-425b-8b4d-98e2771c6791'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:58.259347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] The token cache is being invalidate because project f5b96e92eb674b8e84336c0389bc9b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:44:58.259956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:58.260123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b65d154e-9c9e-425b-8b4d-98e2771c6791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:44:58.261440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 752/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:44:58 2022] DELETE /identity/v3/projects/f5b96e92eb674b8e84336c0389bc9b3f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:45:07.701541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=62 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:45:08.257327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:46:25.543270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8726f2f-ef95-45ab-9b04-fb0016886df7 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:25.543881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8726f2f-ef95-45ab-9b04-fb0016886df7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:25.543881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8726f2f-ef95-45ab-9b04-fb0016886df7 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:25.545397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:25.550619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1337e11-3cd2-44a4-ae53-9181a0d10f85 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:25.551090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1337e11-3cd2-44a4-ae53-9181a0d10f85 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:25.551196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1337e11-3cd2-44a4-ae53-9181a0d10f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:25.637714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e1337e11-3cd2-44a4-ae53-9181a0d10f85 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:25.638907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e1337e11-3cd2-44a4-ae53-9181a0d10f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:25.738960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:25.900066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8726f2f-ef95-45ab-9b04-fb0016886df7 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:25.902270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d8726f2f-ef95-45ab-9b04-fb0016886df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:26.024345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8726f2f-ef95-45ab-9b04-fb0016886df7 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:26.029777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f4b47-adf9-447d-adfe-58c1c402fb7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:26.030382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f4b47-adf9-447d-adfe-58c1c402fb7b service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:26.030660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f4b47-adf9-447d-adfe-58c1c402fb7b service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:26.030939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f4b47-adf9-447d-adfe-58c1c402fb7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:26.038820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f4b47-adf9-447d-adfe-58c1c402fb7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:26.040381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f4b47-adf9-447d-adfe-58c1c402fb7b service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:26.050374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 760/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:46:27.119335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333f0e1-323e-4b5e-a743-315ec4651fc0 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:27.120403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333f0e1-323e-4b5e-a743-315ec4651fc0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:27.120403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333f0e1-323e-4b5e-a743-315ec4651fc0 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:27.121780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:27.127973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d3961-886e-496d-acba-9daa7139666b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:27.128377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d3961-886e-496d-acba-9daa7139666b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:27.128522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d3961-886e-496d-acba-9daa7139666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:27.148582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9c3d3961-886e-496d-acba-9daa7139666b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:27.149736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9c3d3961-886e-496d-acba-9daa7139666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:27.157527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:27.322273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333f0e1-323e-4b5e-a743-315ec4651fc0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:27.336436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333f0e1-323e-4b5e-a743-315ec4651fc0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:27.342425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ae520-e22d-48c0-a99b-642c7c09d3bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:27.343018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ae520-e22d-48c0-a99b-642c7c09d3bb service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:27.343301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ae520-e22d-48c0-a99b-642c7c09d3bb service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:27.343575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ae520-e22d-48c0-a99b-642c7c09d3bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:27.351527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ae520-e22d-48c0-a99b-642c7c09d3bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:27.353123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ae520-e22d-48c0-a99b-642c7c09d3bb service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:27.363095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 755/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:46:27.765530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf8af-d604-4b6e-b744-80f4c1b37952 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:27.765806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf8af-d604-4b6e-b744-80f4c1b37952 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:27.766073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cf8af-d604-4b6e-b744-80f4c1b37952 None None] PATH_INFO: `/v3` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:27.767336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 762/1517] 10.222.0.17 () {54 vars in 925 bytes} [Fri Sep 9 01:46:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:46:27.774063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5748a6-f4ca-4c0e-8e53-294883dc149c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:27.774354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5748a6-f4ca-4c0e-8e53-294883dc149c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:27.774630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5748a6-f4ca-4c0e-8e53-294883dc149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:27.833678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-3a5748a6-f4ca-4c0e-8e53-294883dc149c None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:27.834874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3a5748a6-f4ca-4c0e-8e53-294883dc149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:27.851133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:46:27.907357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 756/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Sep 9 01:46:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:37.090382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49c0dc-dc5c-4fd0-bde6-45163a6f2bda None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:37.090849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49c0dc-dc5c-4fd0-bde6-45163a6f2bda None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:37.090977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49c0dc-dc5c-4fd0-bde6-45163a6f2bda None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:37.092264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 763/1519] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 9 01:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:37.097114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06766d-2613-4ecb-bf8b-75794e83dde5 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:37.097550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06766d-2613-4ecb-bf8b-75794e83dde5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:37.097666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06766d-2613-4ecb-bf8b-75794e83dde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:37.116727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-cb06766d-2613-4ecb-bf8b-75794e83dde5 None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:37.117931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cb06766d-2613-4ecb-bf8b-75794e83dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:37.127073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 757/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 9 01:46:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:37.134584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49c0dc-dc5c-4fd0-bde6-45163a6f2bda None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:37.148244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49c0dc-dc5c-4fd0-bde6-45163a6f2bda None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:37.153008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ffc02-365c-449a-b4b9-e8ca295b1829 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:37.153686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffc02-365c-449a-b4b9-e8ca295b1829 service placement] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:37.153960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffc02-365c-449a-b4b9-e8ca295b1829 service placement] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:37.154223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ffc02-365c-449a-b4b9-e8ca295b1829 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:37.162178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ffc02-365c-449a-b4b9-e8ca295b1829 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:37.163787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ffc02-365c-449a-b4b9-e8ca295b1829 service placement] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:37.173729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 764/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:46:38.842799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02788a25-2f30-4f8f-93e1-543782d625d6 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:38.843345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02788a25-2f30-4f8f-93e1-543782d625d6 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:38.843345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02788a25-2f30-4f8f-93e1-543782d625d6 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:38.844678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:38.850592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2faee-89e4-4c15-a5ae-30b5a440e191 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:38.851023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2faee-89e4-4c15-a5ae-30b5a440e191 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:38.851176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2faee-89e4-4c15-a5ae-30b5a440e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:38.870041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-29a2faee-89e4-4c15-a5ae-30b5a440e191 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:38.871220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-29a2faee-89e4-4c15-a5ae-30b5a440e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:38.879516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:39.034366 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02788a25-2f30-4f8f-93e1-543782d625d6 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:39.048506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02788a25-2f30-4f8f-93e1-543782d625d6 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:39.053820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f0238-a61d-4678-8d02-974ca0fbe831 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:39.054406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f0238-a61d-4678-8d02-974ca0fbe831 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:39.054679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f0238-a61d-4678-8d02-974ca0fbe831 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:39.054945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f0238-a61d-4678-8d02-974ca0fbe831 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:39.063208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f0238-a61d-4678-8d02-974ca0fbe831 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:39.064791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f0238-a61d-4678-8d02-974ca0fbe831 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:39.074470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 759/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:46:40.142583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b01c92-8f7d-4179-8c70-55f202fabeca None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:40.143168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b01c92-8f7d-4179-8c70-55f202fabeca None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:40.143168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b01c92-8f7d-4179-8c70-55f202fabeca None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:40.144481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 01:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:40.150479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f892d-ca51-4e90-80f9-ee75fbca3401 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:40.150871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f892d-ca51-4e90-80f9-ee75fbca3401 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:40.150972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f892d-ca51-4e90-80f9-ee75fbca3401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:40.170576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ca2f892d-ca51-4e90-80f9-ee75fbca3401 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:40.171764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ca2f892d-ca51-4e90-80f9-ee75fbca3401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:40.179618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 01:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:40.331118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b01c92-8f7d-4179-8c70-55f202fabeca None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:40.346011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b01c92-8f7d-4179-8c70-55f202fabeca None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:40.351335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e560ba1-d54d-445e-b1bd-aa8afcc9fec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:40.351917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e560ba1-d54d-445e-b1bd-aa8afcc9fec3 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:40.352192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e560ba1-d54d-445e-b1bd-aa8afcc9fec3 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:40.352461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e560ba1-d54d-445e-b1bd-aa8afcc9fec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:40.360224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e560ba1-d54d-445e-b1bd-aa8afcc9fec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:40.361825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e560ba1-d54d-445e-b1bd-aa8afcc9fec3 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:40.370824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 767/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:46:44.227043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979fe4a-abbd-4240-a35e-6a27066a1a5d None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:44.227659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979fe4a-abbd-4240-a35e-6a27066a1a5d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:44.227659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979fe4a-abbd-4240-a35e-6a27066a1a5d None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:44.228901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:44.234294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f1cd1-436e-4e75-a850-2ff6fb38541c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:44.234744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f1cd1-436e-4e75-a850-2ff6fb38541c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:44.234845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f1cd1-436e-4e75-a850-2ff6fb38541c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:44.254159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6f8f1cd1-436e-4e75-a850-2ff6fb38541c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:44.255331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6f8f1cd1-436e-4e75-a850-2ff6fb38541c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:44.263594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:46:44.549610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979fe4a-abbd-4240-a35e-6a27066a1a5d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:44.551309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0979fe4a-abbd-4240-a35e-6a27066a1a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:44.628541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979fe4a-abbd-4240-a35e-6a27066a1a5d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:44.632035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148dd79-7305-4b4c-b9a5-a3e607b65f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:44.632624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148dd79-7305-4b4c-b9a5-a3e607b65f19 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:44.632901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148dd79-7305-4b4c-b9a5-a3e607b65f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:44.633202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148dd79-7305-4b4c-b9a5-a3e607b65f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:44.642459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148dd79-7305-4b4c-b9a5-a3e607b65f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:44.643869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148dd79-7305-4b4c-b9a5-a3e607b65f19 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:44.654034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 762/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:46:46.785071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29bff9-2c73-49fb-844d-cf007fd04a32 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:46.785651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29bff9-2c73-49fb-844d-cf007fd04a32 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:46.785651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29bff9-2c73-49fb-844d-cf007fd04a32 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:46.786961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:46.792707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac73f0-8217-489f-9b3c-75835625df1e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:46.793159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac73f0-8217-489f-9b3c-75835625df1e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:46.793265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac73f0-8217-489f-9b3c-75835625df1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:46.812708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b6ac73f0-8217-489f-9b3c-75835625df1e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:46.813927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b6ac73f0-8217-489f-9b3c-75835625df1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:46.821931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:47.108874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29bff9-2c73-49fb-844d-cf007fd04a32 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:47.119693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29bff9-2c73-49fb-844d-cf007fd04a32 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:47.123247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371e0d5-60e5-4f06-ad3b-ffc43775c867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:47.123838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e0d5-60e5-4f06-ad3b-ffc43775c867 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:47.124116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e0d5-60e5-4f06-ad3b-ffc43775c867 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:47.124642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371e0d5-60e5-4f06-ad3b-ffc43775c867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:47.133412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371e0d5-60e5-4f06-ad3b-ffc43775c867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:47.135314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371e0d5-60e5-4f06-ad3b-ffc43775c867 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:47.144534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 770/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:46:50.678128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b417b-4ace-4b1e-8f9e-f77f50d4de2c None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:50.678724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b417b-4ace-4b1e-8f9e-f77f50d4de2c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:50.678724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b417b-4ace-4b1e-8f9e-f77f50d4de2c None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:50.680010 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:50.685885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b805f-e342-42b1-b51e-9956cf0f5755 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:50.686300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b805f-e342-42b1-b51e-9956cf0f5755 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:50.686426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b805f-e342-42b1-b51e-9956cf0f5755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:50.693905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=81 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:46:50.706617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-161b805f-e342-42b1-b51e-9956cf0f5755 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:50.707791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-161b805f-e342-42b1-b51e-9956cf0f5755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:50.715465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:51.023313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b417b-4ace-4b1e-8f9e-f77f50d4de2c None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:51.025530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6a0b417b-4ace-4b1e-8f9e-f77f50d4de2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:51.088365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:46:51.103136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b417b-4ace-4b1e-8f9e-f77f50d4de2c None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:51.106675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30128f4c-a915-4656-ba1a-3fa98897ee63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:51.107274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30128f4c-a915-4656-ba1a-3fa98897ee63 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:51.107547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30128f4c-a915-4656-ba1a-3fa98897ee63 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:51.107813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30128f4c-a915-4656-ba1a-3fa98897ee63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:51.116522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30128f4c-a915-4656-ba1a-3fa98897ee63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:51.117974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30128f4c-a915-4656-ba1a-3fa98897ee63 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:51.127894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 765/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:46:53.292559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec10b89-9d50-44f0-af0e-130dc5baad3e None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:53.293140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec10b89-9d50-44f0-af0e-130dc5baad3e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:53.293140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec10b89-9d50-44f0-af0e-130dc5baad3e None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:53.294460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 01:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:46:53.299616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf26e5-9cd3-4039-bbd4-259720713624 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:53.300013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf26e5-9cd3-4039-bbd4-259720713624 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:53.300175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf26e5-9cd3-4039-bbd4-259720713624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:53.320085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-2cbf26e5-9cd3-4039-bbd4-259720713624 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:46:53.321307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2cbf26e5-9cd3-4039-bbd4-259720713624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:53.329738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 01:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:46:53.640841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec10b89-9d50-44f0-af0e-130dc5baad3e None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:46:53.642534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5ec10b89-9d50-44f0-af0e-130dc5baad3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:46:53.655343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec10b89-9d50-44f0-af0e-130dc5baad3e None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:46:53.658851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90180a-c5e9-4733-8428-81e31514a6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:46:53.659447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90180a-c5e9-4733-8428-81e31514a6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:46:53.659724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90180a-c5e9-4733-8428-81e31514a6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:46:53.659995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90180a-c5e9-4733-8428-81e31514a6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:46:53.669242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90180a-c5e9-4733-8428-81e31514a6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:46:53.670666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90180a-c5e9-4733-8428-81e31514a6c6 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:46:53.679869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 773/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:47:03.315264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=11 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:47:03.675760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=8 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:19.420483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7916-a5f3-4c25-9faf-3cee0dfe2130 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.421897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7916-a5f3-4c25-9faf-3cee0dfe2130 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.422300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7916-a5f3-4c25-9faf-3cee0dfe2130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.449006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-c88c7916-a5f3-4c25-9faf-3cee0dfe2130 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.454269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c88c7916-a5f3-4c25-9faf-3cee0dfe2130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.465780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 767/1540] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:48:19.473632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2888755-7d3d-4e3f-a375-08f917affe23 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.475416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2888755-7d3d-4e3f-a375-08f917affe23 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.475988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2888755-7d3d-4e3f-a375-08f917affe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.506264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e2888755-7d3d-4e3f-a375-08f917affe23 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.507852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e2888755-7d3d-4e3f-a375-08f917affe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.516623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 774/1541] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.521924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c7916-a5f3-4c25-9faf-3cee0dfe2130 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:19.534908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c7916-a5f3-4c25-9faf-3cee0dfe2130 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:19.540748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462dfa59-a552-4e65-bab4-4e0d9ca69016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:19.541673 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dfa59-a552-4e65-bab4-4e0d9ca69016 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.542175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dfa59-a552-4e65-bab4-4e0d9ca69016 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.542675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462dfa59-a552-4e65-bab4-4e0d9ca69016 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.543535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dfa59-a552-4e65-bab4-4e0d9ca69016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:19.544054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dfa59-a552-4e65-bab4-4e0d9ca69016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:19.545905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462dfa59-a552-4e65-bab4-4e0d9ca69016 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:19.563052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 768/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:19.571247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2888755-7d3d-4e3f-a375-08f917affe23 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:19.583341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2888755-7d3d-4e3f-a375-08f917affe23 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:19.589418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc931eca-39b8-4947-8dbe-b0ed00cbb4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:19.590313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc931eca-39b8-4947-8dbe-b0ed00cbb4dc admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.590930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc931eca-39b8-4947-8dbe-b0ed00cbb4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.591448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc931eca-39b8-4947-8dbe-b0ed00cbb4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.592313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc931eca-39b8-4947-8dbe-b0ed00cbb4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:19.594490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc931eca-39b8-4947-8dbe-b0ed00cbb4dc admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:19.647405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f39b7-b7bc-4c97-9cf0-680f66368ad2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.647915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f39b7-b7bc-4c97-9cf0-680f66368ad2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.648033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f39b7-b7bc-4c97-9cf0-680f66368ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.649905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 775/1543] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:19.653579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc931eca-39b8-4947-8dbe-b0ed00cbb4dc admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:19.686094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ba9f39b7-b7bc-4c97-9cf0-680f66368ad2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.687318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ba9f39b7-b7bc-4c97-9cf0-680f66368ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.749997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc931eca-39b8-4947-8dbe-b0ed00cbb4dc admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:19.759649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3bcf8-c2ee-42a2-b934-a5dba5f4951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:19.760622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3bcf8-c2ee-42a2-b934-a5dba5f4951f admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.761145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3bcf8-c2ee-42a2-b934-a5dba5f4951f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.761636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3bcf8-c2ee-42a2-b934-a5dba5f4951f admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.762484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3bcf8-c2ee-42a2-b934-a5dba5f4951f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:19.763713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WARNING py.warnings [None req-60e3bcf8-c2ee-42a2-b934-a5dba5f4951f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:48:19.763713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: warnings.warn(msg) Sep 09 01:48:19.763713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: Sep 09 01:48:19.765360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3bcf8-c2ee-42a2-b934-a5dba5f4951f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:19.794194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 776/1544] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:48:19.797471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410256-3728-45d0-9aa0-baf7dd06f845 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.798322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410256-3728-45d0-9aa0-baf7dd06f845 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.799091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410256-3728-45d0-9aa0-baf7dd06f845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.803142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 769/1545] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 09 01:48:19.804530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd64d-245b-40ec-b2c3-fa5c0dbcc82a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.804821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd64d-245b-40ec-b2c3-fa5c0dbcc82a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.805115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd64d-245b-40ec-b2c3-fa5c0dbcc82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.821798 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5c410256-3728-45d0-9aa0-baf7dd06f845 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.823010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5c410256-3728-45d0-9aa0-baf7dd06f845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.826398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b74bd64d-245b-40ec-b2c3-fa5c0dbcc82a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.827558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b74bd64d-245b-40ec-b2c3-fa5c0dbcc82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.831392 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 777/1546] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.832621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8691ab7-8943-4551-8dbe-d59cf23ad74c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.832901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8691ab7-8943-4551-8dbe-d59cf23ad74c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.833239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8691ab7-8943-4551-8dbe-d59cf23ad74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.837185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 770/1547] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.838921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db5471-5358-4388-8dda-fa2bb6cc6349 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.839416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db5471-5358-4388-8dda-fa2bb6cc6349 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.839973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db5471-5358-4388-8dda-fa2bb6cc6349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.853152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c8691ab7-8943-4551-8dbe-d59cf23ad74c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.854353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c8691ab7-8943-4551-8dbe-d59cf23ad74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.860677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-47db5471-5358-4388-8dda-fa2bb6cc6349 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.861858 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-47db5471-5358-4388-8dda-fa2bb6cc6349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.862390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 778/1548] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.863565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2db1fa-43a4-4e28-a959-1a47736e71e0 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.863842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2db1fa-43a4-4e28-a959-1a47736e71e0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.864109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2db1fa-43a4-4e28-a959-1a47736e71e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.877838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 771/1549] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.878729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db5471-5358-4388-8dda-fa2bb6cc6349 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:19.888796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db5471-5358-4388-8dda-fa2bb6cc6349 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:19.890490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-fc2db1fa-43a4-4e28-a959-1a47736e71e0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.891676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-fc2db1fa-43a4-4e28-a959-1a47736e71e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.895305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1342bc6-67b1-42e5-868c-da6ea75d4b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:19.895305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1342bc6-67b1-42e5-868c-da6ea75d4b8b admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.895305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1342bc6-67b1-42e5-868c-da6ea75d4b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.895305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1342bc6-67b1-42e5-868c-da6ea75d4b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.895674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1342bc6-67b1-42e5-868c-da6ea75d4b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:19.896645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1342bc6-67b1-42e5-868c-da6ea75d4b8b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:19.900702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 779/1550] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.901933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578fb7-0ea2-47a3-8566-1bcc2477c1e2 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.902820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578fb7-0ea2-47a3-8566-1bcc2477c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.903104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63578fb7-0ea2-47a3-8566-1bcc2477c1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.912214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 772/1551] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:19.912214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6915e-31bb-40cc-ae9c-534e943eb115 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.912214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6915e-31bb-40cc-ae9c-534e943eb115 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.912214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6915e-31bb-40cc-ae9c-534e943eb115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.925620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-63578fb7-0ea2-47a3-8566-1bcc2477c1e2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.926751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-63578fb7-0ea2-47a3-8566-1bcc2477c1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.933477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6fc6915e-31bb-40cc-ae9c-534e943eb115 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.934636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6fc6915e-31bb-40cc-ae9c-534e943eb115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.935158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 780/1552] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.936341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fa2bf-0aba-450c-94a5-ee2e14dee051 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.936624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fa2bf-0aba-450c-94a5-ee2e14dee051 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.936898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fa2bf-0aba-450c-94a5-ee2e14dee051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.946740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 773/1553] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.948307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ba2b-6513-49b8-9fa9-3a74b12fe899 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.948589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ba2b-6513-49b8-9fa9-3a74b12fe899 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.948854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175ba2b-6513-49b8-9fa9-3a74b12fe899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.958549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2c6fa2bf-0aba-450c-94a5-ee2e14dee051 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.959728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2c6fa2bf-0aba-450c-94a5-ee2e14dee051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.967520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 781/1554] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.968465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0175ba2b-6513-49b8-9fa9-3a74b12fe899 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.968730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aad4bc-559c-4275-ac86-a6ab71866928 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.969010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aad4bc-559c-4275-ac86-a6ab71866928 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.969324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aad4bc-559c-4275-ac86-a6ab71866928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.969640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0175ba2b-6513-49b8-9fa9-3a74b12fe899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.977886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 774/1555] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.979061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86e6d9-9808-4b6e-a7a0-f2a22073f82a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:19.979324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86e6d9-9808-4b6e-a7a0-f2a22073f82a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:19.979594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86e6d9-9808-4b6e-a7a0-f2a22073f82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:19.989214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d8aad4bc-559c-4275-ac86-a6ab71866928 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:19.990950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d8aad4bc-559c-4275-ac86-a6ab71866928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:19.998934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 782/1556] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:19.999217 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7b86e6d9-9808-4b6e-a7a0-f2a22073f82a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:20.000152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ccfbf-feb9-46a5-9e01-3102b9c6cd25 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.000368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7b86e6d9-9808-4b6e-a7a0-f2a22073f82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:20.000489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ccfbf-feb9-46a5-9e01-3102b9c6cd25 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.000688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ccfbf-feb9-46a5-9e01-3102b9c6cd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.008076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 775/1557] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:20.008845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86e6d9-9808-4b6e-a7a0-f2a22073f82a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.018515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86e6d9-9808-4b6e-a7a0-f2a22073f82a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.020504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9f1ccfbf-feb9-46a5-9e01-3102b9c6cd25 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:20.021727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9f1ccfbf-feb9-46a5-9e01-3102b9c6cd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:20.023127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ec5bc-88a0-43dc-bc09-8a9d277a0623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.023707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ec5bc-88a0-43dc-bc09-8a9d277a0623 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.023982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ec5bc-88a0-43dc-bc09-8a9d277a0623 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.024250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ec5bc-88a0-43dc-bc09-8a9d277a0623 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.024871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ec5bc-88a0-43dc-bc09-8a9d277a0623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.026468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ec5bc-88a0-43dc-bc09-8a9d277a0623 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.030324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 783/1558] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:20.031485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506ec3f-a8d9-4d34-bafb-f649237e1c70 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.031760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506ec3f-a8d9-4d34-bafb-f649237e1c70 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.032144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506ec3f-a8d9-4d34-bafb-f649237e1c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.039460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 776/1559] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:20.040314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ec5bc-88a0-43dc-bc09-8a9d277a0623 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.049898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ec5bc-88a0-43dc-bc09-8a9d277a0623 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.054740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2506ec3f-a8d9-4d34-bafb-f649237e1c70 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:20.054952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.055522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.055798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.055903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2506ec3f-a8d9-4d34-bafb-f649237e1c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:20.056091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.056669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:20.056966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.058561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.068264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 784/1560] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:20.068264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506ec3f-a8d9-4d34-bafb-f649237e1c70 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.071238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 777/1561] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:20.072086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.076727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506ec3f-a8d9-4d34-bafb-f649237e1c70 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.081571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.082168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.082447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.082748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.082844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5c374-840c-4461-b24d-f9bd414efdd4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.083399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:20.083804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.084816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WARNING py.warnings [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:48:20.084816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: warnings.warn(msg) Sep 09 01:48:20.084816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: Sep 09 01:48:20.085743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.087514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.088086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.088357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.088624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.089269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:20.089569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.091143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.097573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 785/1562] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:20.098526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.103284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 778/1563] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:20.108662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd2ad7-b714-4c48-811d-c6092e8145b5 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.109718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.113945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.114509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.115369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.115709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.116347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:20.116648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.118272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.120129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a2ac0-b2b3-40c9-9492-335acea01f11 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.124853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.125771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.125771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.125994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.126589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:20.126886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.128423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.130403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 786/1564] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:20.131259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.140543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 779/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:20.141760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.143463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17ae18-b3ef-4e43-966f-63be149b4ec2 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.148212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.148808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.149111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.149388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.150031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:20.150335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.151602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dc49e-37d6-401e-91f0-080387aca5b4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.151946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.156282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.156904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.157224 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.157494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] PATH_INFO: `/v3/projects/288a657ecae14e299870e6d2693d1a16/users/50e1bbe5653844efa2a240f21875b667/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.163974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 787/1566] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:20.170065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.172251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=288a657ecae14e299870e6d2693d1a16, user_id=50e1bbe5653844efa2a240f21875b667, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.173305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING py.warnings [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:48:20.173305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: warnings.warn(msg) Sep 09 01:48:20.173305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: Sep 09 01:48:20.175213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.179982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0ee2-0753-4618-890f-2f2b85d3c103 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.185384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.185967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.186240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.186506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.187158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:20.187457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.188425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 780/1567] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:20 2022] PUT /identity/v3/projects/288a657ecae14e299870e6d2693d1a16/users/50e1bbe5653844efa2a240f21875b667/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:20.189033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.189247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.201776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 788/1568] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:20.202567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.212901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd77a-79ca-4d4d-bcf9-556c5cb5886a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.216454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:20.221636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:20.260974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a9a05-f1e6-4d6b-b865-a4926878b587 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.265866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.266475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.266755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.267028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.267650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.268662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING py.warnings [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:48:20.268662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: warnings.warn(msg) Sep 09 01:48:20.268662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: Sep 09 01:48:20.269929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.273556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef292ae-add2-47a6-bfac-e68a38070bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.274163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef292ae-add2-47a6-bfac-e68a38070bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.274445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef292ae-add2-47a6-bfac-e68a38070bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.274712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef292ae-add2-47a6-bfac-e68a38070bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.275488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef292ae-add2-47a6-bfac-e68a38070bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.277418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef292ae-add2-47a6-bfac-e68a38070bf6 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.299155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 781/1569] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.300087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.307903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 789/1570] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.311563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef292ae-add2-47a6-bfac-e68a38070bf6 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.367114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54797e-14d0-47b5-9341-3a0764726ebd admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.370269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef292ae-add2-47a6-bfac-e68a38070bf6 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.371777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2362f2-32bf-4d0b-b677-f337a9cb1188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.372385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2362f2-32bf-4d0b-b677-f337a9cb1188 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.372663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2362f2-32bf-4d0b-b677-f337a9cb1188 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.372933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2362f2-32bf-4d0b-b677-f337a9cb1188 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.373844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2362f2-32bf-4d0b-b677-f337a9cb1188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.375743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2362f2-32bf-4d0b-b677-f337a9cb1188 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.375992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8612e08-ccbc-4db0-bf9e-795af23d00da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.376595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8612e08-ccbc-4db0-bf9e-795af23d00da admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.376902 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8612e08-ccbc-4db0-bf9e-795af23d00da admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.377216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8612e08-ccbc-4db0-bf9e-795af23d00da admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.377830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8612e08-ccbc-4db0-bf9e-795af23d00da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.379733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8612e08-ccbc-4db0-bf9e-795af23d00da admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.403479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 782/1571] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.404320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2362f2-32bf-4d0b-b677-f337a9cb1188 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.408883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 790/1572] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.409571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8612e08-ccbc-4db0-bf9e-795af23d00da admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.469778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8612e08-ccbc-4db0-bf9e-795af23d00da admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.469972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2362f2-32bf-4d0b-b677-f337a9cb1188 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.474410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e381e-ed24-4147-8718-96dee8a1955c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.475120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57351f01-5519-4c47-ad51-86de4e37cb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.475281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e381e-ed24-4147-8718-96dee8a1955c admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.475281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e381e-ed24-4147-8718-96dee8a1955c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.475554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57351f01-5519-4c47-ad51-86de4e37cb18 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.475670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e381e-ed24-4147-8718-96dee8a1955c admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.475836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57351f01-5519-4c47-ad51-86de4e37cb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.476100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57351f01-5519-4c47-ad51-86de4e37cb18 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.476221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e381e-ed24-4147-8718-96dee8a1955c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.476713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57351f01-5519-4c47-ad51-86de4e37cb18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.478123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e381e-ed24-4147-8718-96dee8a1955c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.478617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57351f01-5519-4c47-ad51-86de4e37cb18 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.504894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 783/1573] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.509498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 791/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.509644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1c310-aa23-4c2d-8521-1b4fa936e619 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.509644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1c310-aa23-4c2d-8521-1b4fa936e619 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.509862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e381e-ed24-4147-8718-96dee8a1955c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.509962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1c310-aa23-4c2d-8521-1b4fa936e619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.513441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:20.535584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-42b1c310-aa23-4c2d-8521-1b4fa936e619 None None] MFA Rules not processed for user `50e1bbe5653844efa2a240f21875b667`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:20.536827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-42b1c310-aa23-4c2d-8521-1b4fa936e619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:20.541773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:20.571799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e381e-ed24-4147-8718-96dee8a1955c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.576646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645d98-2e5b-47b2-81e8-470bcfc416d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.577383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645d98-2e5b-47b2-81e8-470bcfc416d8 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.577766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645d98-2e5b-47b2-81e8-470bcfc416d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.578039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645d98-2e5b-47b2-81e8-470bcfc416d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.578661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60645d98-2e5b-47b2-81e8-470bcfc416d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.580547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60645d98-2e5b-47b2-81e8-470bcfc416d8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.602234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 784/1575] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:20.603115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1c310-aa23-4c2d-8521-1b4fa936e619 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.610635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 792/1576] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.610635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645d98-2e5b-47b2-81e8-470bcfc416d8 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.613087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1c310-aa23-4c2d-8521-1b4fa936e619 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.668997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb374-4cdf-4231-be13-01bddbdb9ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.669632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb374-4cdf-4231-be13-01bddbdb9ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.669902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb374-4cdf-4231-be13-01bddbdb9ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.670172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb374-4cdf-4231-be13-01bddbdb9ec4 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.670831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fb374-4cdf-4231-be13-01bddbdb9ec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.672711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fb374-4cdf-4231-be13-01bddbdb9ec4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.674686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645d98-2e5b-47b2-81e8-470bcfc416d8 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.680156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18257482-caec-4dc4-bd57-1b48b16373ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.680756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18257482-caec-4dc4-bd57-1b48b16373ea admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.681027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18257482-caec-4dc4-bd57-1b48b16373ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.681322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18257482-caec-4dc4-bd57-1b48b16373ea admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.681940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18257482-caec-4dc4-bd57-1b48b16373ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.683983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18257482-caec-4dc4-bd57-1b48b16373ea admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.695648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 785/1577] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.696511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb374-4cdf-4231-be13-01bddbdb9ec4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.706759 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb374-4cdf-4231-be13-01bddbdb9ec4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.713050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 793/1578] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.713050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18257482-caec-4dc4-bd57-1b48b16373ea admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.714786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf241c-53b0-4a37-a6b5-0c5fb531deae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.715428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf241c-53b0-4a37-a6b5-0c5fb531deae admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.715712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf241c-53b0-4a37-a6b5-0c5fb531deae admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.715981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf241c-53b0-4a37-a6b5-0c5fb531deae admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.716593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf241c-53b0-4a37-a6b5-0c5fb531deae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.718510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf241c-53b0-4a37-a6b5-0c5fb531deae admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.720384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18257482-caec-4dc4-bd57-1b48b16373ea admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.725483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bb9dc-df21-4f82-a27c-d7e6fe90eeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.726085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bb9dc-df21-4f82-a27c-d7e6fe90eeef admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.726336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bb9dc-df21-4f82-a27c-d7e6fe90eeef admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.726611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bb9dc-df21-4f82-a27c-d7e6fe90eeef admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.727239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bb9dc-df21-4f82-a27c-d7e6fe90eeef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.729146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511bb9dc-df21-4f82-a27c-d7e6fe90eeef admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.740202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 786/1579] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.741167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf241c-53b0-4a37-a6b5-0c5fb531deae admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.751327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf241c-53b0-4a37-a6b5-0c5fb531deae admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.753029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 794/1580] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/users => generated 303 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.754113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bb9dc-df21-4f82-a27c-d7e6fe90eeef admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.756757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f150b0-0019-4040-a658-6939d3f05ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.757374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f150b0-0019-4040-a658-6939d3f05ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.757652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f150b0-0019-4040-a658-6939d3f05ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.757941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f150b0-0019-4040-a658-6939d3f05ea6 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.758554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f150b0-0019-4040-a658-6939d3f05ea6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.760443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f150b0-0019-4040-a658-6939d3f05ea6 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.766190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511bb9dc-df21-4f82-a27c-d7e6fe90eeef admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.770714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79c481-48d4-4486-8876-b2bf84576e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.771395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c481-48d4-4486-8876-b2bf84576e8e admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.771700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c481-48d4-4486-8876-b2bf84576e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.771975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c481-48d4-4486-8876-b2bf84576e8e admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.772619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79c481-48d4-4486-8876-b2bf84576e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.774853 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79c481-48d4-4486-8876-b2bf84576e8e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.786291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 787/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.786291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f150b0-0019-4040-a658-6939d3f05ea6 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.794988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f150b0-0019-4040-a658-6939d3f05ea6 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.799718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e877-328e-4c29-83fd-bf5e68e0ebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.803238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 795/1582] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.803380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e877-328e-4c29-83fd-bf5e68e0ebad admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.803380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e877-328e-4c29-83fd-bf5e68e0ebad admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.803380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2e877-328e-4c29-83fd-bf5e68e0ebad admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.803654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79c481-48d4-4486-8876-b2bf84576e8e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.803744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2e877-328e-4c29-83fd-bf5e68e0ebad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.803744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2e877-328e-4c29-83fd-bf5e68e0ebad admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.810153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79c481-48d4-4486-8876-b2bf84576e8e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.814883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3b051-d5b9-4ebb-a5f8-b1e4e087b32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.815465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b051-d5b9-4ebb-a5f8-b1e4e087b32c admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.815735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b051-d5b9-4ebb-a5f8-b1e4e087b32c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.815999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3b051-d5b9-4ebb-a5f8-b1e4e087b32c admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.816608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3b051-d5b9-4ebb-a5f8-b1e4e087b32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.819048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da3b051-d5b9-4ebb-a5f8-b1e4e087b32c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.831022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 788/1583] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.832071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e877-328e-4c29-83fd-bf5e68e0ebad admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.842541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 796/1584] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:20.846262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3b051-d5b9-4ebb-a5f8-b1e4e087b32c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.855442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3b051-d5b9-4ebb-a5f8-b1e4e087b32c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.875544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:20.875984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:20.899990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2e877-328e-4c29-83fd-bf5e68e0ebad admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.904439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44696499-12fd-4edf-8194-f31fced2a074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.905049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696499-12fd-4edf-8194-f31fced2a074 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.905374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696499-12fd-4edf-8194-f31fced2a074 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.905646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44696499-12fd-4edf-8194-f31fced2a074 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.906285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44696499-12fd-4edf-8194-f31fced2a074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.907913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44696499-12fd-4edf-8194-f31fced2a074 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.918235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 789/1585] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:20.918983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938d265-b150-48c0-a15f-922d15e6e3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.919599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d265-b150-48c0-a15f-922d15e6e3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.919891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d265-b150-48c0-a15f-922d15e6e3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.920152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d265-b150-48c0-a15f-922d15e6e3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.920892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938d265-b150-48c0-a15f-922d15e6e3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.921663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44696499-12fd-4edf-8194-f31fced2a074 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.922406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938d265-b150-48c0-a15f-922d15e6e3f5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.931205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44696499-12fd-4edf-8194-f31fced2a074 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.936315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 797/1586] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:20.936315 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938d265-b150-48c0-a15f-922d15e6e3f5 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.936558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa8f6e-74e5-4ad6-868d-b5fd98dcde20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.937473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa8f6e-74e5-4ad6-868d-b5fd98dcde20 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.937473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa8f6e-74e5-4ad6-868d-b5fd98dcde20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.937473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa8f6e-74e5-4ad6-868d-b5fd98dcde20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.937915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa8f6e-74e5-4ad6-868d-b5fd98dcde20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.939468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa8f6e-74e5-4ad6-868d-b5fd98dcde20 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.944223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938d265-b150-48c0-a15f-922d15e6e3f5 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.948774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0484a-951c-4555-a2eb-4da36fc62322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.949375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0484a-951c-4555-a2eb-4da36fc62322 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.949649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0484a-951c-4555-a2eb-4da36fc62322 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.949915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0484a-951c-4555-a2eb-4da36fc62322 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.950531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0484a-951c-4555-a2eb-4da36fc62322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.951946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 790/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:20.952097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f0484a-951c-4555-a2eb-4da36fc62322 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.952737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa8f6e-74e5-4ad6-868d-b5fd98dcde20 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.966407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 798/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:20.966407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0484a-951c-4555-a2eb-4da36fc62322 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:20.966663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa8f6e-74e5-4ad6-868d-b5fd98dcde20 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.970588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a68ffc-1ddd-477d-81c5-e5adb96f8d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.971108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a68ffc-1ddd-477d-81c5-e5adb96f8d85 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.971392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a68ffc-1ddd-477d-81c5-e5adb96f8d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.971648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a68ffc-1ddd-477d-81c5-e5adb96f8d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.972289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a68ffc-1ddd-477d-81c5-e5adb96f8d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.973894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a68ffc-1ddd-477d-81c5-e5adb96f8d85 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.975780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0484a-951c-4555-a2eb-4da36fc62322 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:20.980923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02416054-f766-4a28-b956-a7818f84ba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:20.981542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02416054-f766-4a28-b956-a7818f84ba51 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:20.981832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02416054-f766-4a28-b956-a7818f84ba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:20.982078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02416054-f766-4a28-b956-a7818f84ba51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:20.982703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02416054-f766-4a28-b956-a7818f84ba51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:20.984268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02416054-f766-4a28-b956-a7818f84ba51 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:20.984813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 791/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:20.988524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a68ffc-1ddd-477d-81c5-e5adb96f8d85 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.000953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 799/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.000953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02416054-f766-4a28-b956-a7818f84ba51 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.001246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a68ffc-1ddd-477d-81c5-e5adb96f8d85 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.004091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5d6ae-dfd1-4519-a2f6-9c4ea5e87b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.004683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5d6ae-dfd1-4519-a2f6-9c4ea5e87b87 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.004960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5d6ae-dfd1-4519-a2f6-9c4ea5e87b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.005256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5d6ae-dfd1-4519-a2f6-9c4ea5e87b87 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.006447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5d6ae-dfd1-4519-a2f6-9c4ea5e87b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.008337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5d6ae-dfd1-4519-a2f6-9c4ea5e87b87 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.010836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02416054-f766-4a28-b956-a7818f84ba51 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.015419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb16e4-f638-4a19-8fdf-63262c59e51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.015992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb16e4-f638-4a19-8fdf-63262c59e51d admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.016268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb16e4-f638-4a19-8fdf-63262c59e51d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.016533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb16e4-f638-4a19-8fdf-63262c59e51d admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.017197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb16e4-f638-4a19-8fdf-63262c59e51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.018743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb16e4-f638-4a19-8fdf-63262c59e51d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.029675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 800/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.030466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb16e4-f638-4a19-8fdf-63262c59e51d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.032931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 792/1592] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:20 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:21.036582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5d6ae-dfd1-4519-a2f6-9c4ea5e87b87 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.040110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb16e4-f638-4a19-8fdf-63262c59e51d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.044603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d18b81-066b-44ee-bf85-55b7b19f9813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.045209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d18b81-066b-44ee-bf85-55b7b19f9813 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.045483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d18b81-066b-44ee-bf85-55b7b19f9813 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.045750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d18b81-066b-44ee-bf85-55b7b19f9813 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.046076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5d6ae-dfd1-4519-a2f6-9c4ea5e87b87 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.046366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d18b81-066b-44ee-bf85-55b7b19f9813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.048049 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d18b81-066b-44ee-bf85-55b7b19f9813 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.050766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4cfe1-8d0b-48c7-8ef0-cab7e9cd4197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.051398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4cfe1-8d0b-48c7-8ef0-cab7e9cd4197 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.051675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4cfe1-8d0b-48c7-8ef0-cab7e9cd4197 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.051943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4cfe1-8d0b-48c7-8ef0-cab7e9cd4197 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.053083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4cfe1-8d0b-48c7-8ef0-cab7e9cd4197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.054652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4cfe1-8d0b-48c7-8ef0-cab7e9cd4197 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.059158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 801/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.059969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d18b81-066b-44ee-bf85-55b7b19f9813 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.065222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 793/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.069504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4cfe1-8d0b-48c7-8ef0-cab7e9cd4197 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.070001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d18b81-066b-44ee-bf85-55b7b19f9813 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.074629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff250c0-38f0-4c26-8d01-2588f16fcb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.075650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff250c0-38f0-4c26-8d01-2588f16fcb06 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.075650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff250c0-38f0-4c26-8d01-2588f16fcb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.075882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff250c0-38f0-4c26-8d01-2588f16fcb06 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.076364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff250c0-38f0-4c26-8d01-2588f16fcb06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.078013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff250c0-38f0-4c26-8d01-2588f16fcb06 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.078935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4cfe1-8d0b-48c7-8ef0-cab7e9cd4197 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.084040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d4e42-a6d3-4add-ab14-f89343bfe9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.084623 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d4e42-a6d3-4add-ab14-f89343bfe9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.084898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d4e42-a6d3-4add-ab14-f89343bfe9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.085206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d4e42-a6d3-4add-ab14-f89343bfe9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.085827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d4e42-a6d3-4add-ab14-f89343bfe9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.087381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d4e42-a6d3-4add-ab14-f89343bfe9a1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.088475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 802/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.089286 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff250c0-38f0-4c26-8d01-2588f16fcb06 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.098744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff250c0-38f0-4c26-8d01-2588f16fcb06 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.101249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 794/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.102110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d4e42-a6d3-4add-ab14-f89343bfe9a1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.104413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052be22-776a-47a7-9f2f-469a037b55a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.104991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052be22-776a-47a7-9f2f-469a037b55a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.105306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052be22-776a-47a7-9f2f-469a037b55a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.105580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052be22-776a-47a7-9f2f-469a037b55a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.106204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052be22-776a-47a7-9f2f-469a037b55a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.107768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052be22-776a-47a7-9f2f-469a037b55a9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.111671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d4e42-a6d3-4add-ab14-f89343bfe9a1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.116237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5cc37-5960-4c42-96fa-88850b9600b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.116804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5cc37-5960-4c42-96fa-88850b9600b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.117104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5cc37-5960-4c42-96fa-88850b9600b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.117378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5cc37-5960-4c42-96fa-88850b9600b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.117997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5cc37-5960-4c42-96fa-88850b9600b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.118475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 803/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.119275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052be22-776a-47a7-9f2f-469a037b55a9 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.119677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5cc37-5960-4c42-96fa-88850b9600b0 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.129185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052be22-776a-47a7-9f2f-469a037b55a9 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.134067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 795/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.134067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5cc37-5960-4c42-96fa-88850b9600b0 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.134300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbce8ae-27ee-4526-b604-d9353a6994b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.134611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbce8ae-27ee-4526-b604-d9353a6994b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.134891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbce8ae-27ee-4526-b604-d9353a6994b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.135162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbce8ae-27ee-4526-b604-d9353a6994b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.135788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbce8ae-27ee-4526-b604-d9353a6994b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.137398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbce8ae-27ee-4526-b604-d9353a6994b1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.141324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5cc37-5960-4c42-96fa-88850b9600b0 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.145863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe4ee5-d925-4dea-99e8-913eb6e8af62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.146962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe4ee5-d925-4dea-99e8-913eb6e8af62 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.147237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe4ee5-d925-4dea-99e8-913eb6e8af62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.147503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe4ee5-d925-4dea-99e8-913eb6e8af62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.148229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe4ee5-d925-4dea-99e8-913eb6e8af62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.148351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 804/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.149014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbce8ae-27ee-4526-b604-d9353a6994b1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.149920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe4ee5-d925-4dea-99e8-913eb6e8af62 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.158223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbce8ae-27ee-4526-b604-d9353a6994b1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.160864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 796/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.161946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe4ee5-d925-4dea-99e8-913eb6e8af62 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.162836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311df47-6f0f-436e-ad51-b7c88747d45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.163468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311df47-6f0f-436e-ad51-b7c88747d45b admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.163747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311df47-6f0f-436e-ad51-b7c88747d45b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.164020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311df47-6f0f-436e-ad51-b7c88747d45b admin admin] PATH_INFO: `/v3/projects/89dfb75234034ebb8cae1e5d30d7fff1/users/70f7fc0917664b5495198480392cdffb/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.172174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe4ee5-d925-4dea-99e8-913eb6e8af62 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.176709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af73989-d44b-4a86-9a3e-164b5bd0a880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.177850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af73989-d44b-4a86-9a3e-164b5bd0a880 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.178121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af73989-d44b-4a86-9a3e-164b5bd0a880 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.178386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af73989-d44b-4a86-9a3e-164b5bd0a880 admin admin] PATH_INFO: `/v3/projects/940a7e4459874ae3b54c5db3ec096b8d/users/6d415854b71b4366a2f8d11e64271ccc/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.179577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311df47-6f0f-436e-ad51-b7c88747d45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89dfb75234034ebb8cae1e5d30d7fff1, user_id=70f7fc0917664b5495198480392cdffb, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.182388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311df47-6f0f-436e-ad51-b7c88747d45b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.193758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af73989-d44b-4a86-9a3e-164b5bd0a880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940a7e4459874ae3b54c5db3ec096b8d, user_id=6d415854b71b4366a2f8d11e64271ccc, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.196321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 805/1601] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:21 2022] PUT /identity/v3/projects/89dfb75234034ebb8cae1e5d30d7fff1/users/70f7fc0917664b5495198480392cdffb/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:21.196482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af73989-d44b-4a86-9a3e-164b5bd0a880 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.197130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311df47-6f0f-436e-ad51-b7c88747d45b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.215196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 797/1602] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:21 2022] PUT /identity/v3/projects/940a7e4459874ae3b54c5db3ec096b8d/users/6d415854b71b4366a2f8d11e64271ccc/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:21.216035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af73989-d44b-4a86-9a3e-164b5bd0a880 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.266122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311df47-6f0f-436e-ad51-b7c88747d45b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.278665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af73989-d44b-4a86-9a3e-164b5bd0a880 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.283524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ad842-5884-47c9-bc26-c5ca45bfd299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.284188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ad842-5884-47c9-bc26-c5ca45bfd299 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.284469 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ad842-5884-47c9-bc26-c5ca45bfd299 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.284745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ad842-5884-47c9-bc26-c5ca45bfd299 admin admin] PATH_INFO: `/v3/projects/64d07b0088404243bd360cbe75f8e87b/users/34eaed625c7e4fe9adfe636a9e713a7c/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.289242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:21.299800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ad842-5884-47c9-bc26-c5ca45bfd299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d07b0088404243bd360cbe75f8e87b, user_id=34eaed625c7e4fe9adfe636a9e713a7c, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.302986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ad842-5884-47c9-bc26-c5ca45bfd299 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.317680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 798/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:21 2022] PUT /identity/v3/projects/64d07b0088404243bd360cbe75f8e87b/users/34eaed625c7e4fe9adfe636a9e713a7c/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:21.328900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ad842-5884-47c9-bc26-c5ca45bfd299 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.329024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58b73a-8410-4a3f-935f-8151ce1ca480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.329024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b73a-8410-4a3f-935f-8151ce1ca480 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.329024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b73a-8410-4a3f-935f-8151ce1ca480 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.329024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b73a-8410-4a3f-935f-8151ce1ca480 admin admin] PATH_INFO: `/v3/projects/6d01372b1bdb420fbdbda8d44285fbbe/users/98e198600f974feaaa0e62a23d690f25/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.332133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:21.342272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58b73a-8410-4a3f-935f-8151ce1ca480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d01372b1bdb420fbdbda8d44285fbbe, user_id=98e198600f974feaaa0e62a23d690f25, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.344914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58b73a-8410-4a3f-935f-8151ce1ca480 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.357614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 806/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:21 2022] PUT /identity/v3/projects/6d01372b1bdb420fbdbda8d44285fbbe/users/98e198600f974feaaa0e62a23d690f25/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:21.361942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58b73a-8410-4a3f-935f-8151ce1ca480 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.385996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ad842-5884-47c9-bc26-c5ca45bfd299 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.425887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58b73a-8410-4a3f-935f-8151ce1ca480 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.431130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbb335-5d9e-4ae4-988b-3f814658ec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.431787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbb335-5d9e-4ae4-988b-3f814658ec43 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.432062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbb335-5d9e-4ae4-988b-3f814658ec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.432340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbb335-5d9e-4ae4-988b-3f814658ec43 admin admin] PATH_INFO: `/v3/projects/6b35000faf844d758c12de9938c235a8/users/3153c9ff011f4ab8958fc49795e6f8f1/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.440791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6eb5f-ac0e-42d7-ace8-d6b91bab7a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.441473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6eb5f-ac0e-42d7-ace8-d6b91bab7a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.441753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6eb5f-ac0e-42d7-ace8-d6b91bab7a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.442027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6eb5f-ac0e-42d7-ace8-d6b91bab7a9c admin admin] PATH_INFO: `/v3/projects/1ed13a0f61d34a80898093dc1d8e6610/users/83f9dc8c41604f0e86eaf57202285cc4/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.446735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbb335-5d9e-4ae4-988b-3f814658ec43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b35000faf844d758c12de9938c235a8, user_id=3153c9ff011f4ab8958fc49795e6f8f1, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.449414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbb335-5d9e-4ae4-988b-3f814658ec43 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.456731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6eb5f-ac0e-42d7-ace8-d6b91bab7a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ed13a0f61d34a80898093dc1d8e6610, user_id=83f9dc8c41604f0e86eaf57202285cc4, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.459408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6eb5f-ac0e-42d7-ace8-d6b91bab7a9c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.467915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 807/1605] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:21 2022] PUT /identity/v3/projects/6b35000faf844d758c12de9938c235a8/users/3153c9ff011f4ab8958fc49795e6f8f1/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:21.468814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbb335-5d9e-4ae4-988b-3f814658ec43 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.472672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 799/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:21 2022] PUT /identity/v3/projects/1ed13a0f61d34a80898093dc1d8e6610/users/83f9dc8c41604f0e86eaf57202285cc4/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:21.473760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6eb5f-ac0e-42d7-ace8-d6b91bab7a9c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.531618 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbb335-5d9e-4ae4-988b-3f814658ec43 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.534684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6eb5f-ac0e-42d7-ace8-d6b91bab7a9c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.537146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8af5e-281c-4b25-9509-f753084c9af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.537841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8af5e-281c-4b25-9509-f753084c9af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.538110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8af5e-281c-4b25-9509-f753084c9af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.538381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8af5e-281c-4b25-9509-f753084c9af8 admin admin] PATH_INFO: `/v3/projects/daa9c339892144d281422ad480383a22/users/c8f3463f4c534fa7b2a87e5cac6c3a70/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.539733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6f499-8150-4ff2-86aa-5b81b2db09ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.540394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6f499-8150-4ff2-86aa-5b81b2db09ad admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.540651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6f499-8150-4ff2-86aa-5b81b2db09ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.540917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6f499-8150-4ff2-86aa-5b81b2db09ad admin admin] PATH_INFO: `/v3/projects/44ac7ae9cfb249318eaab14aa6c7973a/users/a30ee97d3471468080e9914c6b7740dd/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.553703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8af5e-281c-4b25-9509-f753084c9af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa9c339892144d281422ad480383a22, user_id=c8f3463f4c534fa7b2a87e5cac6c3a70, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.555741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6f499-8150-4ff2-86aa-5b81b2db09ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ac7ae9cfb249318eaab14aa6c7973a, user_id=a30ee97d3471468080e9914c6b7740dd, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.556355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8af5e-281c-4b25-9509-f753084c9af8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.558437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6f499-8150-4ff2-86aa-5b81b2db09ad admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.569920 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 808/1607] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:21 2022] PUT /identity/v3/projects/daa9c339892144d281422ad480383a22/users/c8f3463f4c534fa7b2a87e5cac6c3a70/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:21.571204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723d573-7102-43c3-a7b1-c149445a4946 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.571524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723d573-7102-43c3-a7b1-c149445a4946 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.575090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 800/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:21 2022] PUT /identity/v3/projects/44ac7ae9cfb249318eaab14aa6c7973a/users/a30ee97d3471468080e9914c6b7740dd/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:21.575227 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723d573-7102-43c3-a7b1-c149445a4946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.575310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34792622-62e0-420f-9177-c4878df46533 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.575310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34792622-62e0-420f-9177-c4878df46533 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.575310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34792622-62e0-420f-9177-c4878df46533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.593331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-34792622-62e0-420f-9177-c4878df46533 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:21.594529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-34792622-62e0-420f-9177-c4878df46533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:21.599790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-8723d573-7102-43c3-a7b1-c149445a4946 None None] MFA Rules not processed for user `70f7fc0917664b5495198480392cdffb`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:21.600967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8723d573-7102-43c3-a7b1-c149445a4946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:21.604825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:21.614130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:21.656399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 801/1609] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:21.657867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f35c2c-f562-45db-9270-73e36389c4b5 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.658222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f35c2c-f562-45db-9270-73e36389c4b5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.658560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f35c2c-f562-45db-9270-73e36389c4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.672696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 809/1610] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:21.674076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677d53c-6adb-4eb5-ab13-f34b8db4e7fa None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.674423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677d53c-6adb-4eb5-ab13-f34b8db4e7fa None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.674755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677d53c-6adb-4eb5-ab13-f34b8db4e7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.695061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b3f35c2c-f562-45db-9270-73e36389c4b5 None None] MFA Rules not processed for user `34eaed625c7e4fe9adfe636a9e713a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:21.696275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b3f35c2c-f562-45db-9270-73e36389c4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:21.701417 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2677d53c-6adb-4eb5-ab13-f34b8db4e7fa None None] MFA Rules not processed for user `98e198600f974feaaa0e62a23d690f25`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:21.702570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2677d53c-6adb-4eb5-ab13-f34b8db4e7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:21.765421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 802/1611] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:21.766437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 810/1612] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:21.767033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f839dd3-d3e7-4955-b847-0d29350eac80 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.767317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f839dd3-d3e7-4955-b847-0d29350eac80 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.767609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f839dd3-d3e7-4955-b847-0d29350eac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.768362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f00501-0c8e-4ac9-8f40-90b88b295ea0 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.768799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f00501-0c8e-4ac9-8f40-90b88b295ea0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.769257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f00501-0c8e-4ac9-8f40-90b88b295ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.787002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0f839dd3-d3e7-4955-b847-0d29350eac80 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:21.788082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0f839dd3-d3e7-4955-b847-0d29350eac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:21.788676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e8f00501-0c8e-4ac9-8f40-90b88b295ea0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:21.790344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e8f00501-0c8e-4ac9-8f40-90b88b295ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:21.797349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 803/1613] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:21.801091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 811/1614] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:21.801091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765b9e-15de-4c30-92d5-aa378fee89d5 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.801091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765b9e-15de-4c30-92d5-aa378fee89d5 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.801091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765b9e-15de-4c30-92d5-aa378fee89d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.802041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f839dd3-d3e7-4955-b847-0d29350eac80 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.813356 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f839dd3-d3e7-4955-b847-0d29350eac80 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.819424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5eb9c9-d8b0-43ec-ab3a-fd364ea7399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.820132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5eb9c9-d8b0-43ec-ab3a-fd364ea7399e admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.820530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5eb9c9-d8b0-43ec-ab3a-fd364ea7399e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.820928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5eb9c9-d8b0-43ec-ab3a-fd364ea7399e admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.821616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5eb9c9-d8b0-43ec-ab3a-fd364ea7399e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.823432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5eb9c9-d8b0-43ec-ab3a-fd364ea7399e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.831584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-58765b9e-15de-4c30-92d5-aa378fee89d5 None None] MFA Rules not processed for user `c8f3463f4c534fa7b2a87e5cac6c3a70`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:21.832878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-58765b9e-15de-4c30-92d5-aa378fee89d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:21.839018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 804/1615] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:21.839018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5eb9c9-d8b0-43ec-ab3a-fd364ea7399e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.898282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5eb9c9-d8b0-43ec-ab3a-fd364ea7399e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.902189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 812/1616] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:21.902874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58765b9e-15de-4c30-92d5-aa378fee89d5 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.903884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fd7a3-250f-4f78-af88-3867e7b53491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.904919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fd7a3-250f-4f78-af88-3867e7b53491 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.905130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fd7a3-250f-4f78-af88-3867e7b53491 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.905472 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fd7a3-250f-4f78-af88-3867e7b53491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.914569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58765b9e-15de-4c30-92d5-aa378fee89d5 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.922172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0758a-171a-4842-ba6e-39d90dc5a972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.922904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0758a-171a-4842-ba6e-39d90dc5a972 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.923250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0758a-171a-4842-ba6e-39d90dc5a972 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.923589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0758a-171a-4842-ba6e-39d90dc5a972 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.923990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fd7a3-250f-4f78-af88-3867e7b53491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.924361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0758a-171a-4842-ba6e-39d90dc5a972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.925643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fd7a3-250f-4f78-af88-3867e7b53491 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.926460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0758a-171a-4842-ba6e-39d90dc5a972 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.936919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 805/1617] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:21.937721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fd7a3-250f-4f78-af88-3867e7b53491 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.946879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:21.950811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fd7a3-250f-4f78-af88-3867e7b53491 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:21.956176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:21.956898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.957333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.957706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.958475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:21.960497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:21.964097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 813/1618] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:21.964097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a327817-613d-4203-886c-471fc926c8a5 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:21.964097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a327817-613d-4203-886c-471fc926c8a5 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:21.964097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a327817-613d-4203-886c-471fc926c8a5 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:21.965619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 814/1619] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 9 01:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:48:21.970367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a327817-613d-4203-886c-471fc926c8a5 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.972919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2a327817-613d-4203-886c-471fc926c8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:21.992311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 806/1620] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:21 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:21.993428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:21.995259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:22.020177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:22.049819 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a327817-613d-4203-886c-471fc926c8a5 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.064935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac60d2a-3986-4211-a172-a585a4a8c5ea admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.069916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca28556-e02a-4d53-8548-ef7508779ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.070531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca28556-e02a-4d53-8548-ef7508779ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.071144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca28556-e02a-4d53-8548-ef7508779ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.071418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca28556-e02a-4d53-8548-ef7508779ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.104281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3562605-911d-4d7d-9bab-e4625cf52825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.104896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3562605-911d-4d7d-9bab-e4625cf52825 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.105229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3562605-911d-4d7d-9bab-e4625cf52825 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.105623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3562605-911d-4d7d-9bab-e4625cf52825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.144251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca28556-e02a-4d53-8548-ef7508779ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.145994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca28556-e02a-4d53-8548-ef7508779ab0 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.158000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 807/1621] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:22.170003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca28556-e02a-4d53-8548-ef7508779ab0 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.183933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3562605-911d-4d7d-9bab-e4625cf52825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.185974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3562605-911d-4d7d-9bab-e4625cf52825 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.192288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca28556-e02a-4d53-8548-ef7508779ab0 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.203084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade42ff-60ab-4fd6-8fe5-f61f26fd80e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.204202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade42ff-60ab-4fd6-8fe5-f61f26fd80e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.204614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade42ff-60ab-4fd6-8fe5-f61f26fd80e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.205259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 815/1622] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:22.205653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade42ff-60ab-4fd6-8fe5-f61f26fd80e7 admin admin] PATH_INFO: `/v3/projects/44ac7ae9cfb249318eaab14aa6c7973a/users/a30ee97d3471468080e9914c6b7740dd/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.206671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0efa-e822-480b-80fc-4d3c8b418817 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.207022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0efa-e822-480b-80fc-4d3c8b418817 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.207305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca0efa-e822-480b-80fc-4d3c8b418817 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.208456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade42ff-60ab-4fd6-8fe5-f61f26fd80e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ac7ae9cfb249318eaab14aa6c7973a, user_id=a30ee97d3471468080e9914c6b7740dd, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.208596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 816/1623] 10.222.0.15 () {54 vars in 962 bytes} [Fri Sep 9 01:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:48:22.211508 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade42ff-60ab-4fd6-8fe5-f61f26fd80e7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.213705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca0efa-e822-480b-80fc-4d3c8b418817 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.226344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 808/1624] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:22 2022] PUT /identity/v3/projects/44ac7ae9cfb249318eaab14aa6c7973a/users/a30ee97d3471468080e9914c6b7740dd/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:22.227990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6e29-1609-4ccf-87d9-e00ae29ad5a8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.228379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6e29-1609-4ccf-87d9-e00ae29ad5a8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.228757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a6e29-1609-4ccf-87d9-e00ae29ad5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.236654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca0efa-e822-480b-80fc-4d3c8b418817 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.278593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4d6a6e29-1609-4ccf-87d9-e00ae29ad5a8 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:22.280611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4d6a6e29-1609-4ccf-87d9-e00ae29ad5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:22.331561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542279b3-5568-4e2f-9894-5a8f3ccc83a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.332268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542279b3-5568-4e2f-9894-5a8f3ccc83a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.332550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542279b3-5568-4e2f-9894-5a8f3ccc83a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.332947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542279b3-5568-4e2f-9894-5a8f3ccc83a1 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.333710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542279b3-5568-4e2f-9894-5a8f3ccc83a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.335766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542279b3-5568-4e2f-9894-5a8f3ccc83a1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.357785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:22.380813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 817/1625] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:22 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:22.380813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542279b3-5568-4e2f-9894-5a8f3ccc83a1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.389924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 809/1626] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 9 01:48:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:22.392538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a998596-55ca-4319-a9ef-b70ed38ef87d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.392759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542279b3-5568-4e2f-9894-5a8f3ccc83a1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.393297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a998596-55ca-4319-a9ef-b70ed38ef87d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.393805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a998596-55ca-4319-a9ef-b70ed38ef87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.398131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132124e2-d6e8-4a7d-b199-907c001eb135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.398775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132124e2-d6e8-4a7d-b199-907c001eb135 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.399284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132124e2-d6e8-4a7d-b199-907c001eb135 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.399571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132124e2-d6e8-4a7d-b199-907c001eb135 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.400295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132124e2-d6e8-4a7d-b199-907c001eb135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.402678 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132124e2-d6e8-4a7d-b199-907c001eb135 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.421959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4a998596-55ca-4319-a9ef-b70ed38ef87d None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:22.424213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4a998596-55ca-4319-a9ef-b70ed38ef87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:22.432850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 818/1627] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:22 2022] POST /identity/v3/users => generated 308 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:22.434262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4d1f-6734-4ac6-9abf-72442ce5a0e2 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.434552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4d1f-6734-4ac6-9abf-72442ce5a0e2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.434861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d4d1f-6734-4ac6-9abf-72442ce5a0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.438869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 810/1628] 10.222.0.15 () {58 vars in 1059 bytes} [Fri Sep 9 01:48:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:22.449361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a998596-55ca-4319-a9ef-b70ed38ef87d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.466577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a998596-55ca-4319-a9ef-b70ed38ef87d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.476106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46c929-97f3-443f-8ee3-df95b37dbe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.476906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46c929-97f3-443f-8ee3-df95b37dbe36 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.477248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46c929-97f3-443f-8ee3-df95b37dbe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.477421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6b0d4d1f-6734-4ac6-9abf-72442ce5a0e2 None None] MFA Rules not processed for user `a30ee97d3471468080e9914c6b7740dd`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:22.477544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46c929-97f3-443f-8ee3-df95b37dbe36 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.478199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46c929-97f3-443f-8ee3-df95b37dbe36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.479379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6b0d4d1f-6734-4ac6-9abf-72442ce5a0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:22.480581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46c929-97f3-443f-8ee3-df95b37dbe36 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.497662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 811/1629] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:22.498971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46c929-97f3-443f-8ee3-df95b37dbe36 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.516664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46c929-97f3-443f-8ee3-df95b37dbe36 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.525361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8ea4f-532e-472d-99bf-5d072f461989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.526917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8ea4f-532e-472d-99bf-5d072f461989 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.528061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8ea4f-532e-472d-99bf-5d072f461989 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.529652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8ea4f-532e-472d-99bf-5d072f461989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.553585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:22.597598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 819/1630] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:22.598797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d4d1f-6734-4ac6-9abf-72442ce5a0e2 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.621193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d4d1f-6734-4ac6-9abf-72442ce5a0e2 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.629589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d3376-eb7c-4aa9-a949-a33b01e176be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.630413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d3376-eb7c-4aa9-a949-a33b01e176be admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.631241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d3376-eb7c-4aa9-a949-a33b01e176be admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.631583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d3376-eb7c-4aa9-a949-a33b01e176be admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.632449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d3376-eb7c-4aa9-a949-a33b01e176be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.634333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d3376-eb7c-4aa9-a949-a33b01e176be admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.650657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8ea4f-532e-472d-99bf-5d072f461989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.651580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 820/1631] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:22.654728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8ea4f-532e-472d-99bf-5d072f461989 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.657604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d3376-eb7c-4aa9-a949-a33b01e176be admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.670440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 812/1632] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:22.671384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8ea4f-532e-472d-99bf-5d072f461989 service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.676468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d3376-eb7c-4aa9-a949-a33b01e176be admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.683326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8ea4f-532e-472d-99bf-5d072f461989 service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.684437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e228f-d229-47d7-9de0-8254ca13a7c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.685412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e228f-d229-47d7-9de0-8254ca13a7c0 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.686108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e228f-d229-47d7-9de0-8254ca13a7c0 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.686472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e228f-d229-47d7-9de0-8254ca13a7c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.689434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15404a-7b4e-4030-a1a7-4ca1818752bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.690109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15404a-7b4e-4030-a1a7-4ca1818752bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.690390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15404a-7b4e-4030-a1a7-4ca1818752bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.690719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15404a-7b4e-4030-a1a7-4ca1818752bd admin admin] PATH_INFO: `/v3/projects/85238ef62b4943b29c925e3bfc77d7c8/users/b0c62176ec2042c7a590bdf8b0317ec6/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.725009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15404a-7b4e-4030-a1a7-4ca1818752bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=85238ef62b4943b29c925e3bfc77d7c8, user_id=b0c62176ec2042c7a590bdf8b0317ec6, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.730629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15404a-7b4e-4030-a1a7-4ca1818752bd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.756392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 813/1633] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:22 2022] PUT /identity/v3/projects/85238ef62b4943b29c925e3bfc77d7c8/users/b0c62176ec2042c7a590bdf8b0317ec6/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:22.757680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15404a-7b4e-4030-a1a7-4ca1818752bd admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.764359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e228f-d229-47d7-9de0-8254ca13a7c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:22.766993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e228f-d229-47d7-9de0-8254ca13a7c0 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:22.779724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 821/1634] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:22.780831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e228f-d229-47d7-9de0-8254ca13a7c0 service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:22.824710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e228f-d229-47d7-9de0-8254ca13a7c0 service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.842576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15404a-7b4e-4030-a1a7-4ca1818752bd admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:22.847387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6fcd7-5d65-4297-bc88-37712468dba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.848015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6fcd7-5d65-4297-bc88-37712468dba6 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.848296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6fcd7-5d65-4297-bc88-37712468dba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.848567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6fcd7-5d65-4297-bc88-37712468dba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.939619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:22.964929 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e43b0e-396d-4746-85e8-560b0980b1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:22.966823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e43b0e-396d-4746-85e8-560b0980b1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:22.967422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e43b0e-396d-4746-85e8-560b0980b1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:22.968314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e43b0e-396d-4746-85e8-560b0980b1f8 admin admin] PATH_INFO: `/v3/projects/b29836dff48d4466a866d41c92739d9f/users/91a3263ad63a49aeaeaa1ed6eab2e2f3/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:22.995903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e43b0e-396d-4746-85e8-560b0980b1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29836dff48d4466a866d41c92739d9f, user_id=91a3263ad63a49aeaeaa1ed6eab2e2f3, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:23.003383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e43b0e-396d-4746-85e8-560b0980b1f8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:23.024845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 822/1635] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:22 2022] PUT /identity/v3/projects/b29836dff48d4466a866d41c92739d9f/users/91a3263ad63a49aeaeaa1ed6eab2e2f3/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:23.028507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e43b0e-396d-4746-85e8-560b0980b1f8 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:23.032479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6fcd7-5d65-4297-bc88-37712468dba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:23.038024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6fcd7-5d65-4297-bc88-37712468dba6 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:23.063538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 814/1636] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:23.064594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6fcd7-5d65-4297-bc88-37712468dba6 service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:23.092238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6fcd7-5d65-4297-bc88-37712468dba6 service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:23.147014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e43b0e-396d-4746-85e8-560b0980b1f8 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:23.155087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb148d-cbee-40a8-a687-623171dfcb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:23.156046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb148d-cbee-40a8-a687-623171dfcb99 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.156888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb148d-cbee-40a8-a687-623171dfcb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.157640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb148d-cbee-40a8-a687-623171dfcb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.158531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb148d-cbee-40a8-a687-623171dfcb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:23.160830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb148d-cbee-40a8-a687-623171dfcb99 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:23.176599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 823/1637] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:23.182050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb148d-cbee-40a8-a687-623171dfcb99 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:23.195601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb148d-cbee-40a8-a687-623171dfcb99 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:23.206114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1c43b-ac3c-4e55-b728-4baf333ab9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:23.209628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1c43b-ac3c-4e55-b728-4baf333ab9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.210930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:23.212967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1c43b-ac3c-4e55-b728-4baf333ab9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.213709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1c43b-ac3c-4e55-b728-4baf333ab9f3 admin admin] PATH_INFO: `/v3/projects/85238ef62b4943b29c925e3bfc77d7c8/users/b0c62176ec2042c7a590bdf8b0317ec6/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.220277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1c43b-ac3c-4e55-b728-4baf333ab9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85238ef62b4943b29c925e3bfc77d7c8, user_id=b0c62176ec2042c7a590bdf8b0317ec6, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:23.230869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1c43b-ac3c-4e55-b728-4baf333ab9f3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:23.238566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501daa7-3a37-48fc-8581-dc313dd3580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:23.239184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501daa7-3a37-48fc-8581-dc313dd3580c admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.239467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501daa7-3a37-48fc-8581-dc313dd3580c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.240293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501daa7-3a37-48fc-8581-dc313dd3580c admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.241048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501daa7-3a37-48fc-8581-dc313dd3580c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:23.242591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501daa7-3a37-48fc-8581-dc313dd3580c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:23.259680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 824/1638] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:23 2022] PUT /identity/v3/projects/85238ef62b4943b29c925e3bfc77d7c8/users/b0c62176ec2042c7a590bdf8b0317ec6/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:23.270558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5ce35-59bd-47ec-9791-2dad3b88914a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.270558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5ce35-59bd-47ec-9791-2dad3b88914a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.271434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5ce35-59bd-47ec-9791-2dad3b88914a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.280263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 815/1639] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:23.281581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501daa7-3a37-48fc-8581-dc313dd3580c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:23.312521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501daa7-3a37-48fc-8581-dc313dd3580c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:23.331616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-98b5ce35-59bd-47ec-9791-2dad3b88914a None None] MFA Rules not processed for user `b0c62176ec2042c7a590bdf8b0317ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:23.335937 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-98b5ce35-59bd-47ec-9791-2dad3b88914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:23.448158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81059b-0799-4b82-bcaa-d8f13644180f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:23.450261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81059b-0799-4b82-bcaa-d8f13644180f admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.451052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81059b-0799-4b82-bcaa-d8f13644180f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.451330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81059b-0799-4b82-bcaa-d8f13644180f admin admin] PATH_INFO: `/v3/projects/b29836dff48d4466a866d41c92739d9f/users/91a3263ad63a49aeaeaa1ed6eab2e2f3/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.455526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81059b-0799-4b82-bcaa-d8f13644180f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29836dff48d4466a866d41c92739d9f, user_id=91a3263ad63a49aeaeaa1ed6eab2e2f3, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:23.461018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81059b-0799-4b82-bcaa-d8f13644180f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:23.491151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 816/1640] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:23 2022] PUT /identity/v3/projects/b29836dff48d4466a866d41c92739d9f/users/91a3263ad63a49aeaeaa1ed6eab2e2f3/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:23.504910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232212c-aa28-4b75-b8d0-887ad09c2e2e None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.504910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232212c-aa28-4b75-b8d0-887ad09c2e2e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.505855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 825/1641] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:23.505948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232212c-aa28-4b75-b8d0-887ad09c2e2e None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.506044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e9ad3-5813-4c9d-8f6e-c7bc1bbebc8f None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.506044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e9ad3-5813-4c9d-8f6e-c7bc1bbebc8f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.506044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e9ad3-5813-4c9d-8f6e-c7bc1bbebc8f None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.506323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 817/1642] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 9 01:48:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:48:23.506416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 826/1643] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 9 01:48:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:48:23.511550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec75c9-2b41-460f-ab96-08d10fecbc95 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.511840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec75c9-2b41-460f-ab96-08d10fecbc95 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.512114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec75c9-2b41-460f-ab96-08d10fecbc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.516482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89228075-b570-46a7-8caf-24d5993d0df6 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.516482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89228075-b570-46a7-8caf-24d5993d0df6 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.516482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89228075-b570-46a7-8caf-24d5993d0df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.574329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-89228075-b570-46a7-8caf-24d5993d0df6 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:23.579504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-89228075-b570-46a7-8caf-24d5993d0df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:23.592031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-64ec75c9-2b41-460f-ab96-08d10fecbc95 None None] MFA Rules not processed for user `91a3263ad63a49aeaeaa1ed6eab2e2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:23.595613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-64ec75c9-2b41-460f-ab96-08d10fecbc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:23.687342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:23.782511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 818/1644] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:23 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:23.782511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec75c9-2b41-460f-ab96-08d10fecbc95 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:23.785356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 827/1645] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:48:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:23.789406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57b2d1-71a5-43ff-a72f-c6ad81552738 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.791120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57b2d1-71a5-43ff-a72f-c6ad81552738 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.791521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57b2d1-71a5-43ff-a72f-c6ad81552738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:23.846451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ac57b2d1-71a5-43ff-a72f-c6ad81552738 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:23.852170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ac57b2d1-71a5-43ff-a72f-c6ad81552738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:23.897362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 828/1646] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:48:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:23.933555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec75c9-2b41-460f-ab96-08d10fecbc95 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:23.960695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb85fb1-df3e-47b6-8cdd-af7d20b06184 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:23.961451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb85fb1-df3e-47b6-8cdd-af7d20b06184 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:23.961799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb85fb1-df3e-47b6-8cdd-af7d20b06184 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:23.962138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb85fb1-df3e-47b6-8cdd-af7d20b06184 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:24.005156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57b2d1-71a5-43ff-a72f-c6ad81552738 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:24.018918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57b2d1-71a5-43ff-a72f-c6ad81552738 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:24.023855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:24.031857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec465d4-cef8-441f-aa92-174880835ffc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:24.032962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec465d4-cef8-441f-aa92-174880835ffc service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:24.033577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec465d4-cef8-441f-aa92-174880835ffc service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:24.034019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec465d4-cef8-441f-aa92-174880835ffc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:24.060514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec465d4-cef8-441f-aa92-174880835ffc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:24.062381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec465d4-cef8-441f-aa92-174880835ffc service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:24.088476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 829/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:24 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:24.151065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb85fb1-df3e-47b6-8cdd-af7d20b06184 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:24.152609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb85fb1-df3e-47b6-8cdd-af7d20b06184 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:24.173552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 819/1648] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:24.266262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec465d4-cef8-441f-aa92-174880835ffc service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:24.279853 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec465d4-cef8-441f-aa92-174880835ffc service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:24.290440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f938c-38ab-42ee-b811-6f2512f6d319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:24.292226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f938c-38ab-42ee-b811-6f2512f6d319 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:24.292527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f938c-38ab-42ee-b811-6f2512f6d319 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:24.292808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f938c-38ab-42ee-b811-6f2512f6d319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:24.324408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f938c-38ab-42ee-b811-6f2512f6d319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:24.325960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f938c-38ab-42ee-b811-6f2512f6d319 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:24.332693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb85fb1-df3e-47b6-8cdd-af7d20b06184 service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:24.342683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb85fb1-df3e-47b6-8cdd-af7d20b06184 service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:24.350596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 830/1649] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:24.355120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c887f-590f-4aca-aab5-36e992cd9e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:24.355748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c887f-590f-4aca-aab5-36e992cd9e26 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:24.356369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c887f-590f-4aca-aab5-36e992cd9e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:24.356655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c887f-590f-4aca-aab5-36e992cd9e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:24.381962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c887f-590f-4aca-aab5-36e992cd9e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:24.383552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c887f-590f-4aca-aab5-36e992cd9e26 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:24.412308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 820/1650] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:25.695161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f938c-38ab-42ee-b811-6f2512f6d319 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:25.952275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f938c-38ab-42ee-b811-6f2512f6d319 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:25.971661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:25.990209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:25.995195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:25.995869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] PATH_INFO: `/v3/users/34eaed625c7e4fe9adfe636a9e713a7c` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:26.001266 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=34eaed625c7e4fe9adfe636a9e713a7c)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:26.017667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:26.149843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34eaed625c7e4fe9adfe636a9e713a7c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:26.154176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34eaed625c7e4fe9adfe636a9e713a7c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:26.186415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34eaed625c7e4fe9adfe636a9e713a7c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:26.218358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34eaed625c7e4fe9adfe636a9e713a7c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:26.219046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34eaed625c7e4fe9adfe636a9e713a7c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:26.249046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 831/1651] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:25 2022] DELETE /identity/v3/users/34eaed625c7e4fe9adfe636a9e713a7c => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:26.253459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2878de-680b-4b40-9496-d5716768e098 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:26.254011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2878de-680b-4b40-9496-d5716768e098 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:26.254011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2878de-680b-4b40-9496-d5716768e098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:26.293320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0b2878de-680b-4b40-9496-d5716768e098 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:26.294559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0b2878de-680b-4b40-9496-d5716768e098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:26.502255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 821/1652] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:26.548531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:26.550691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:26.563829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2878de-680b-4b40-9496-d5716768e098 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:26.565620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0b2878de-680b-4b40-9496-d5716768e098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:26.613961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2878de-680b-4b40-9496-d5716768e098 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:26.635080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:26.640833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:26.641484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:26.641793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:26.642040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] PATH_INFO: `/v3/users/70f7fc0917664b5495198480392cdffb` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:26.645095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f7fc0917664b5495198480392cdffb)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:26.645974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: WARNING py.warnings [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:48:26.645974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: warnings.warn(msg) Sep 09 01:48:26.645974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: Sep 09 01:48:26.647201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:26.838285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0dcbd-b2ba-4d0f-9425-3d8e630929cc admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:26.855693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ea768c-38e8-4b55-8a97-4927475a0e20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:26.857555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea768c-38e8-4b55-8a97-4927475a0e20 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:26.858195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea768c-38e8-4b55-8a97-4927475a0e20 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:26.858470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea768c-38e8-4b55-8a97-4927475a0e20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:26.887144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea768c-38e8-4b55-8a97-4927475a0e20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:26.888737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ea768c-38e8-4b55-8a97-4927475a0e20 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:26.922201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f7fc0917664b5495198480392cdffb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:26.923701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f7fc0917664b5495198480392cdffb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:26.934123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 832/1653] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:26.939811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f7fc0917664b5495198480392cdffb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:26.979466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f7fc0917664b5495198480392cdffb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:27.035058 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f7fc0917664b5495198480392cdffb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:27.065655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 822/1654] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:26 2022] DELETE /identity/v3/users/70f7fc0917664b5495198480392cdffb => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:27.069676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774530-eb39-4ba3-a715-71a955d091c0 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:27.070124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774530-eb39-4ba3-a715-71a955d091c0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:27.070403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774530-eb39-4ba3-a715-71a955d091c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:27.115342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4f774530-eb39-4ba3-a715-71a955d091c0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:27.118657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4f774530-eb39-4ba3-a715-71a955d091c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:27.227810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 833/1655] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:27.353250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:27.355222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:27.373812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:27.487135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f28c7-2744-4618-b24b-14fb9b1c1bf7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:27.492256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:27.493661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:27.493958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:27.494235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:27.508765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:27.510455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:27.527174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 823/1656] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:28.439597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774530-eb39-4ba3-a715-71a955d091c0 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:28.441516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4f774530-eb39-4ba3-a715-71a955d091c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:28.450466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:28.452099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:28.465824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774530-eb39-4ba3-a715-71a955d091c0 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:28.473437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7209d0-7e0d-454f-be00-1379d0149976 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:28.492475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:28.493365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:28.493796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:28.494219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] PATH_INFO: `/v3/projects/89dfb75234034ebb8cae1e5d30d7fff1` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:28.496727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89dfb75234034ebb8cae1e5d30d7fff1)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:28.498497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: WARNING py.warnings [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 01:48:28.498497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: warnings.warn(msg) Sep 09 01:48:28.498497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: Sep 09 01:48:28.500311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:28.507268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:28.508150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:28.508590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:28.509012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] PATH_INFO: `/v3/projects/64d07b0088404243bd360cbe75f8e87b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:28.513584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d07b0088404243bd360cbe75f8e87b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:28.518625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:28.684403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89dfb75234034ebb8cae1e5d30d7fff1', 'request_id': 'req-5077ee43-13d5-42cd-b539-5c7a46a206b2'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.685809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d07b0088404243bd360cbe75f8e87b', 'request_id': 'req-1e213353-3ec2-487b-8210-f0ee8a1513ec'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.698109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d07b0088404243bd360cbe75f8e87b', 'request_id': 'req-1e213353-3ec2-487b-8210-f0ee8a1513ec'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.715080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d07b0088404243bd360cbe75f8e87b', 'request_id': 'req-1e213353-3ec2-487b-8210-f0ee8a1513ec'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.724620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89dfb75234034ebb8cae1e5d30d7fff1', 'request_id': 'req-5077ee43-13d5-42cd-b539-5c7a46a206b2'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.737820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] The token cache is being invalidate because project 64d07b0088404243bd360cbe75f8e87b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:28.738984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.742638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.743718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89dfb75234034ebb8cae1e5d30d7fff1', 'request_id': 'req-5077ee43-13d5-42cd-b539-5c7a46a206b2'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.750103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 824/1657] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:28 2022] DELETE /identity/v3/projects/64d07b0088404243bd360cbe75f8e87b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:48:28.752270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:28.755274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:28.780350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] The token cache is being invalidate because project 89dfb75234034ebb8cae1e5d30d7fff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:28.780641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.798461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:28.800372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 834/1658] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:28 2022] DELETE /identity/v3/projects/89dfb75234034ebb8cae1e5d30d7fff1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:28.804973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:28.806586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:28.923882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:28.966059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077ee43-13d5-42cd-b539-5c7a46a206b2 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:28.977192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e213353-3ec2-487b-8210-f0ee8a1513ec admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:28.981364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:28.982259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:28.982738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:28.983209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] PATH_INFO: `/v3/users/b0c62176ec2042c7a590bdf8b0317ec6` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:28.988947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0c62176ec2042c7a590bdf8b0317ec6)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:28.991755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:28.994687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:28.999326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:28.999614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:28.999897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] PATH_INFO: `/v3/users/91a3263ad63a49aeaeaa1ed6eab2e2f3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:29.001677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a3263ad63a49aeaeaa1ed6eab2e2f3)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:29.006950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:29.093622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0c62176ec2042c7a590bdf8b0317ec6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.093817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:29.099584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0c62176ec2042c7a590bdf8b0317ec6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.119998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a3263ad63a49aeaeaa1ed6eab2e2f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.121436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a3263ad63a49aeaeaa1ed6eab2e2f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.122247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0c62176ec2042c7a590bdf8b0317ec6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.141687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a3263ad63a49aeaeaa1ed6eab2e2f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.147494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0c62176ec2042c7a590bdf8b0317ec6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.148624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0c62176ec2042c7a590bdf8b0317ec6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.170060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 835/1659] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:28 2022] DELETE /identity/v3/users/b0c62176ec2042c7a590bdf8b0317ec6 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:48:29.170968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a3263ad63a49aeaeaa1ed6eab2e2f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.191255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb6ae55f-38ce-4318-9773-624a6f908d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a3263ad63a49aeaeaa1ed6eab2e2f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:29.211232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 825/1660] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:28 2022] DELETE /identity/v3/users/91a3263ad63a49aeaeaa1ed6eab2e2f3 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:29.794282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:29.795744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:29.982260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591b923-9639-4a0d-90c1-357a7b10ae1f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:30.082932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:30.084971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:30.085638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:30.086092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] PATH_INFO: `/v3/projects/b29836dff48d4466a866d41c92739d9f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:30.087778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29836dff48d4466a866d41c92739d9f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:30.090290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:30.105346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:30.215331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29836dff48d4466a866d41c92739d9f', 'request_id': 'req-ac165f66-cc02-4e99-b38b-4632b29c13fd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:30.222732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29836dff48d4466a866d41c92739d9f', 'request_id': 'req-ac165f66-cc02-4e99-b38b-4632b29c13fd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:30.236943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29836dff48d4466a866d41c92739d9f', 'request_id': 'req-ac165f66-cc02-4e99-b38b-4632b29c13fd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:30.285824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] The token cache is being invalidate because project b29836dff48d4466a866d41c92739d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:30.287498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:30.290853 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac165f66-cc02-4e99-b38b-4632b29c13fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:30.297960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528f9d-5d96-44ff-81f9-5a55c1ff2a9a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:30.298428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528f9d-5d96-44ff-81f9-5a55c1ff2a9a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:30.298600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528f9d-5d96-44ff-81f9-5a55c1ff2a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:30.302729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 836/1661] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:29 2022] DELETE /identity/v3/projects/b29836dff48d4466a866d41c92739d9f => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:30.330104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a7528f9d-5d96-44ff-81f9-5a55c1ff2a9a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:30.331829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a7528f9d-5d96-44ff-81f9-5a55c1ff2a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:30.490580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d134-2616-4364-86e1-54857c0e16aa None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:30.491379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d134-2616-4364-86e1-54857c0e16aa None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:30.491756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9d134-2616-4364-86e1-54857c0e16aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:30.495481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 826/1662] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:30.544284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e9b9d134-2616-4364-86e1-54857c0e16aa None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:30.548194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e9b9d134-2616-4364-86e1-54857c0e16aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:30.586297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7528f9d-5d96-44ff-81f9-5a55c1ff2a9a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:30.599741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 837/1663] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:30.601104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7528f9d-5d96-44ff-81f9-5a55c1ff2a9a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:30.618970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:30.619783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:30.620157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:30.620526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:30.621389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:30.621793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:30.624641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:30.647867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9d134-2616-4364-86e1-54857c0e16aa None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:30.660781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 827/1664] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:30.661940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9d134-2616-4364-86e1-54857c0e16aa None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:30.670454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d772e-1343-4d98-9509-761a0ecc1c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:30.672290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d772e-1343-4d98-9509-761a0ecc1c8e admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:30.672669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d772e-1343-4d98-9509-761a0ecc1c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:30.673768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d772e-1343-4d98-9509-761a0ecc1c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:30.675030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d772e-1343-4d98-9509-761a0ecc1c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:30.677357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d772e-1343-4d98-9509-761a0ecc1c8e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:30.734657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 838/1665] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:30 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:30.736138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:30.749468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c116a-1e28-4c29-b4f6-28f810f11d0d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:30.926429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e90972-93f6-4441-9ea8-90e9d3650815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:30.928148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90972-93f6-4441-9ea8-90e9d3650815 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:30.928651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90972-93f6-4441-9ea8-90e9d3650815 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:30.929162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90972-93f6-4441-9ea8-90e9d3650815 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:30.930023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e90972-93f6-4441-9ea8-90e9d3650815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:30.932194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e90972-93f6-4441-9ea8-90e9d3650815 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:30.987026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d772e-1343-4d98-9509-761a0ecc1c8e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:30.989906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 828/1666] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:30 2022] POST /identity/v3/users => generated 290 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:48:31.013135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d772e-1343-4d98-9509-761a0ecc1c8e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:31.036769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:31.037453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.038466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.038739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.039409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:31.042776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:31.103866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 839/1667] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:31.104799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e90972-93f6-4441-9ea8-90e9d3650815 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:31.106665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:31.118223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:31.118435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e90972-93f6-4441-9ea8-90e9d3650815 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:31.124522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66fc5-eecd-4e1b-b59b-aa2c2f76a38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:31.125179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66fc5-eecd-4e1b-b59b-aa2c2f76a38e admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.125467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66fc5-eecd-4e1b-b59b-aa2c2f76a38e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.125863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f66fc5-eecd-4e1b-b59b-aa2c2f76a38e admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.126521 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f66fc5-eecd-4e1b-b59b-aa2c2f76a38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:31.128334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f66fc5-eecd-4e1b-b59b-aa2c2f76a38e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:31.142316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 829/1668] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:31.146238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66fc5-eecd-4e1b-b59b-aa2c2f76a38e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:31.158150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f66fc5-eecd-4e1b-b59b-aa2c2f76a38e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:31.161648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20833f52-f2ac-4133-8ed4-ffea4e362706 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:31.164913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c885772-13da-4b2d-9cea-3c5711ae1a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:31.166747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885772-13da-4b2d-9cea-3c5711ae1a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.167321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885772-13da-4b2d-9cea-3c5711ae1a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.168231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c885772-13da-4b2d-9cea-3c5711ae1a47 admin admin] PATH_INFO: `/v3/projects/88a0a47943c74230b45151abca2271e5/users/df3d135051b14af78ddb93494a2119c0/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.173279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:31.173938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.174215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.174486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] PATH_INFO: `/v3/projects/85238ef62b4943b29c925e3bfc77d7c8` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.175650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85238ef62b4943b29c925e3bfc77d7c8)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:31.178215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:31.217225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c885772-13da-4b2d-9cea-3c5711ae1a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a0a47943c74230b45151abca2271e5, user_id=df3d135051b14af78ddb93494a2119c0, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:31.226864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c885772-13da-4b2d-9cea-3c5711ae1a47 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:31.234518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:31.260498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 830/1669] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:31 2022] PUT /identity/v3/projects/88a0a47943c74230b45151abca2271e5/users/df3d135051b14af78ddb93494a2119c0/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:31.270971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642374d-56a2-4419-a595-792066ffeae0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.271381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642374d-56a2-4419-a595-792066ffeae0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.271804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642374d-56a2-4419-a595-792066ffeae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.297786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85238ef62b4943b29c925e3bfc77d7c8', 'request_id': 'req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:31.314123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85238ef62b4943b29c925e3bfc77d7c8', 'request_id': 'req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:31.323946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85238ef62b4943b29c925e3bfc77d7c8', 'request_id': 'req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:31.344250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] The token cache is being invalidate because project 85238ef62b4943b29c925e3bfc77d7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:31.344556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:31.345282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2bd4f0ce-0d08-4e81-9d1e-2a4a7ae38b84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:31.346348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4642374d-56a2-4419-a595-792066ffeae0 None None] MFA Rules not processed for user `df3d135051b14af78ddb93494a2119c0`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:31.347115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 840/1670] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:31 2022] DELETE /identity/v3/projects/85238ef62b4943b29c925e3bfc77d7c8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:31.352422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00cca-5226-452c-93c5-161aa4de9ce3 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.352550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4642374d-56a2-4419-a595-792066ffeae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:31.352702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00cca-5226-452c-93c5-161aa4de9ce3 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.353037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00cca-5226-452c-93c5-161aa4de9ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.407283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c9d00cca-5226-452c-93c5-161aa4de9ce3 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:31.409354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c9d00cca-5226-452c-93c5-161aa4de9ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:31.499085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:31.513781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 831/1671] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:48:31.518786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642374d-56a2-4419-a595-792066ffeae0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:31.521656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4642374d-56a2-4419-a595-792066ffeae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:31.554260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 841/1672] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:48:31.555871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd650a-7c12-43d1-a9f6-cb3048f1e60f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.556745 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd650a-7c12-43d1-a9f6-cb3048f1e60f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.557948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd650a-7c12-43d1-a9f6-cb3048f1e60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.598882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e9bd650a-7c12-43d1-a9f6-cb3048f1e60f None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:31.601346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e9bd650a-7c12-43d1-a9f6-cb3048f1e60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:31.621813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 842/1673] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:31.627097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd650a-7c12-43d1-a9f6-cb3048f1e60f None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:31.642648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd650a-7c12-43d1-a9f6-cb3048f1e60f None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:31.663958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:31.664778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.665165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.665565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.666350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:31.667729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:31.669982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:31.674280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642374d-56a2-4419-a595-792066ffeae0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:31.689750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bf5d2-07f5-4d76-b5c9-ba9d0fc3e223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:31.690529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bf5d2-07f5-4d76-b5c9-ba9d0fc3e223 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.691048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bf5d2-07f5-4d76-b5c9-ba9d0fc3e223 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.691390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bf5d2-07f5-4d76-b5c9-ba9d0fc3e223 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.692125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9bf5d2-07f5-4d76-b5c9-ba9d0fc3e223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:31.694322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9bf5d2-07f5-4d76-b5c9-ba9d0fc3e223 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:31.704393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 843/1674] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:31.705504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:31.724504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1498f04-a339-4a08-9496-e2617c8c0052 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:31.735524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f5cdf-af07-4b84-8379-620f96626ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:31.736604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5cdf-af07-4b84-8379-620f96626ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:31.736604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5cdf-af07-4b84-8379-620f96626ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:31.737022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5cdf-af07-4b84-8379-620f96626ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:31.737775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f5cdf-af07-4b84-8379-620f96626ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:31.744545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f5cdf-af07-4b84-8379-620f96626ad1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:31.756118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 832/1675] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:31 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:31.762514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bf5d2-07f5-4d76-b5c9-ba9d0fc3e223 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:31.826695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 844/1676] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:31 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:31.827925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f5cdf-af07-4b84-8379-620f96626ad1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:31.961187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bf5d2-07f5-4d76-b5c9-ba9d0fc3e223 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.009207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:32.033803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f5cdf-af07-4b84-8379-620f96626ad1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.056649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b5ae2-a60f-4d38-b86a-3c73a20bd25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.057472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5ae2-a60f-4d38-b86a-3c73a20bd25a admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.057850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5ae2-a60f-4d38-b86a-3c73a20bd25a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.058217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5ae2-a60f-4d38-b86a-3c73a20bd25a admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.059071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b5ae2-a60f-4d38-b86a-3c73a20bd25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.070834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b5ae2-a60f-4d38-b86a-3c73a20bd25a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.145885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 845/1677] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:31 2022] POST /identity/v3/users => generated 292 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:32.146931 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b5ae2-a60f-4d38-b86a-3c73a20bd25a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:32.193628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b5ae2-a60f-4d38-b86a-3c73a20bd25a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.194505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6efbd-d60d-4503-aa15-3e498b6c37bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.196686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6efbd-d60d-4503-aa15-3e498b6c37bf admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.198441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6efbd-d60d-4503-aa15-3e498b6c37bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.198441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6efbd-d60d-4503-aa15-3e498b6c37bf admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.199325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6efbd-d60d-4503-aa15-3e498b6c37bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.202914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6efbd-d60d-4503-aa15-3e498b6c37bf admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.212228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8e720-9575-4c31-a53b-38e6cf388e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.213010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e720-9575-4c31-a53b-38e6cf388e42 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.213431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e720-9575-4c31-a53b-38e6cf388e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.213803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8e720-9575-4c31-a53b-38e6cf388e42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.219317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8e720-9575-4c31-a53b-38e6cf388e42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.221468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8e720-9575-4c31-a53b-38e6cf388e42 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.243058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 846/1678] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:32.249991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8e720-9575-4c31-a53b-38e6cf388e42 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:32.264251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 833/1679] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:31 2022] POST /identity/v3/users => generated 297 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:32.267489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6efbd-d60d-4503-aa15-3e498b6c37bf admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:32.282119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8e720-9575-4c31-a53b-38e6cf388e42 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.288919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6efbd-d60d-4503-aa15-3e498b6c37bf admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.292743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1914f0-89ce-40d3-9948-7ff06b4a1131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.293981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1914f0-89ce-40d3-9948-7ff06b4a1131 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.295022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1914f0-89ce-40d3-9948-7ff06b4a1131 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.295719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1914f0-89ce-40d3-9948-7ff06b4a1131 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.297224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1914f0-89ce-40d3-9948-7ff06b4a1131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.302267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1914f0-89ce-40d3-9948-7ff06b4a1131 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.302418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceea462-1c8f-4f0c-a916-02b2060fce32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.302418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceea462-1c8f-4f0c-a916-02b2060fce32 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.302785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceea462-1c8f-4f0c-a916-02b2060fce32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.303095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceea462-1c8f-4f0c-a916-02b2060fce32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.303814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceea462-1c8f-4f0c-a916-02b2060fce32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.306211 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceea462-1c8f-4f0c-a916-02b2060fce32 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.324901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 847/1680] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:32.332874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1914f0-89ce-40d3-9948-7ff06b4a1131 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:32.335072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 834/1681] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:32.339506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceea462-1c8f-4f0c-a916-02b2060fce32 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:32.367608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceea462-1c8f-4f0c-a916-02b2060fce32 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.383828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285131c0-6fcc-4813-a0e9-f986a57a176d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.383828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285131c0-6fcc-4813-a0e9-f986a57a176d admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.383828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285131c0-6fcc-4813-a0e9-f986a57a176d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.383828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285131c0-6fcc-4813-a0e9-f986a57a176d admin admin] PATH_INFO: `/v3/projects/591cdb1b657a41c5b9f2cb500fbfd58d/users/5918220422994787ac84f9681a391c0a/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.384785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1914f0-89ce-40d3-9948-7ff06b4a1131 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.394123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7af1df-2c0c-4c36-a916-41b0745c2767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.396722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7af1df-2c0c-4c36-a916-41b0745c2767 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.397152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7af1df-2c0c-4c36-a916-41b0745c2767 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.399309 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7af1df-2c0c-4c36-a916-41b0745c2767 admin admin] PATH_INFO: `/v3/projects/6be3300e9b3d4c30ab20b8995187c499/users/fe7be079e4754539874fe6a989d3ce28/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.403850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285131c0-6fcc-4813-a0e9-f986a57a176d admin admin] RBAC: Authorizing `identity:create_grant(project_id=591cdb1b657a41c5b9f2cb500fbfd58d, user_id=5918220422994787ac84f9681a391c0a, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.407214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285131c0-6fcc-4813-a0e9-f986a57a176d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.443757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 835/1682] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:32 2022] PUT /identity/v3/projects/591cdb1b657a41c5b9f2cb500fbfd58d/users/5918220422994787ac84f9681a391c0a/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:32.450714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7af1df-2c0c-4c36-a916-41b0745c2767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be3300e9b3d4c30ab20b8995187c499, user_id=fe7be079e4754539874fe6a989d3ce28, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.451414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a68cf-8876-490a-91a6-dfdb15e4f9c4 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.451590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a68cf-8876-490a-91a6-dfdb15e4f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.452960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a68cf-8876-490a-91a6-dfdb15e4f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.459835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7af1df-2c0c-4c36-a916-41b0745c2767 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.486471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 848/1683] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:32 2022] PUT /identity/v3/projects/6be3300e9b3d4c30ab20b8995187c499/users/fe7be079e4754539874fe6a989d3ce28/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:32.487353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7af1df-2c0c-4c36-a916-41b0745c2767 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:32.522769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0b4a68cf-8876-490a-91a6-dfdb15e4f9c4 None None] MFA Rules not processed for user `6d415854b71b4366a2f8d11e64271ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:32.524014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0b4a68cf-8876-490a-91a6-dfdb15e4f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:32.589990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:32.655194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7af1df-2c0c-4c36-a916-41b0745c2767 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.677323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76b862-d232-4770-aec7-4b6f5a65f3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.678714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b862-d232-4770-aec7-4b6f5a65f3ec admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.681107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b862-d232-4770-aec7-4b6f5a65f3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.681476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76b862-d232-4770-aec7-4b6f5a65f3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.683997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76b862-d232-4770-aec7-4b6f5a65f3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.686652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76b862-d232-4770-aec7-4b6f5a65f3ec admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.725992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 849/1684] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:32.736348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99b7c3-4d71-436a-8a01-df7f92773dfa None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.736640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99b7c3-4d71-436a-8a01-df7f92773dfa None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.736918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99b7c3-4d71-436a-8a01-df7f92773dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.742290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 836/1685] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:32.743232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a68cf-8876-490a-91a6-dfdb15e4f9c4 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:32.780767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a68cf-8876-490a-91a6-dfdb15e4f9c4 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:32.791163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd13762-ff30-47ba-9239-d49eae847f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:32.791902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd13762-ff30-47ba-9239-d49eae847f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.792354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd13762-ff30-47ba-9239-d49eae847f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.792635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd13762-ff30-47ba-9239-d49eae847f6a admin admin] PATH_INFO: `/v3/projects/591cdb1b657a41c5b9f2cb500fbfd58d/users/5918220422994787ac84f9681a391c0a/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.794756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd13762-ff30-47ba-9239-d49eae847f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=591cdb1b657a41c5b9f2cb500fbfd58d, user_id=5918220422994787ac84f9681a391c0a, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:32.797773 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd13762-ff30-47ba-9239-d49eae847f6a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:32.810865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 837/1686] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:32 2022] PUT /identity/v3/projects/591cdb1b657a41c5b9f2cb500fbfd58d/users/5918220422994787ac84f9681a391c0a/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:32.812150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a505e-2429-4cf9-acb7-1c31f30c813c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.812624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a505e-2429-4cf9-acb7-1c31f30c813c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.812903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a505e-2429-4cf9-acb7-1c31f30c813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:32.822499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2c99b7c3-4d71-436a-8a01-df7f92773dfa None None] MFA Rules not processed for user `fe7be079e4754539874fe6a989d3ce28`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:32.823765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2c99b7c3-4d71-436a-8a01-df7f92773dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:32.842184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-5f7a505e-2429-4cf9-acb7-1c31f30c813c None None] MFA Rules not processed for user `6d415854b71b4366a2f8d11e64271ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:32.845401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5f7a505e-2429-4cf9-acb7-1c31f30c813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:32.959580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:32.981943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 838/1687] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:32.983552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57103f-cee8-4448-9aa5-a33576c2a41a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:32.984039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57103f-cee8-4448-9aa5-a33576c2a41a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:32.985414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57103f-cee8-4448-9aa5-a33576c2a41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:33.006551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 850/1688] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:33.009475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99b7c3-4d71-436a-8a01-df7f92773dfa None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:33.015383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2c99b7c3-4d71-436a-8a01-df7f92773dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:33.067735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8c57103f-cee8-4448-9aa5-a33576c2a41a None None] MFA Rules not processed for user `5918220422994787ac84f9681a391c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:33.069044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8c57103f-cee8-4448-9aa5-a33576c2a41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:33.141844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c99b7c3-4d71-436a-8a01-df7f92773dfa None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:33.151319 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3243801-20c2-418c-8bf2-c3c928ab73a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:33.152256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3243801-20c2-418c-8bf2-c3c928ab73a7 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:33.152256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3243801-20c2-418c-8bf2-c3c928ab73a7 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:33.152514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3243801-20c2-418c-8bf2-c3c928ab73a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:33.167975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3243801-20c2-418c-8bf2-c3c928ab73a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:33.170003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3243801-20c2-418c-8bf2-c3c928ab73a7 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:33.201208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 851/1689] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:33.202098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3243801-20c2-418c-8bf2-c3c928ab73a7 service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:33.248911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 839/1690] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:33.264617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57103f-cee8-4448-9aa5-a33576c2a41a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:33.266212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8c57103f-cee8-4448-9aa5-a33576c2a41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:33.358252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:33.422859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3243801-20c2-418c-8bf2-c3c928ab73a7 service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:33.441747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0baeb6-9238-4abd-a6cf-45062d78629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:33.442927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0baeb6-9238-4abd-a6cf-45062d78629c admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:33.443215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0baeb6-9238-4abd-a6cf-45062d78629c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:33.443491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0baeb6-9238-4abd-a6cf-45062d78629c admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:33.444144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0baeb6-9238-4abd-a6cf-45062d78629c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:33.450950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0baeb6-9238-4abd-a6cf-45062d78629c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:33.527093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57103f-cee8-4448-9aa5-a33576c2a41a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:33.532733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e662b3b-600c-4efa-8043-c9549b800d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:33.533528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e662b3b-600c-4efa-8043-c9549b800d96 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:33.535388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e662b3b-600c-4efa-8043-c9549b800d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:33.535778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e662b3b-600c-4efa-8043-c9549b800d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:33.552464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 852/1691] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:33 2022] POST /identity/v3/projects => generated 385 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:33.556087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0baeb6-9238-4abd-a6cf-45062d78629c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:33.576466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e662b3b-600c-4efa-8043-c9549b800d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:33.578600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e662b3b-600c-4efa-8043-c9549b800d96 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:33.629304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 840/1692] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:33.781806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e662b3b-600c-4efa-8043-c9549b800d96 service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:33.782720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0baeb6-9238-4abd-a6cf-45062d78629c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:33.796331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e662b3b-600c-4efa-8043-c9549b800d96 service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:33.809626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d76631-7f72-42bc-9ca7-9f7d609dffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:33.812126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d76631-7f72-42bc-9ca7-9f7d609dffc2 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:33.812516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d76631-7f72-42bc-9ca7-9f7d609dffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:33.812832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d76631-7f72-42bc-9ca7-9f7d609dffc2 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:33.814576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d76631-7f72-42bc-9ca7-9f7d609dffc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:33.823771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d76631-7f72-42bc-9ca7-9f7d609dffc2 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:33.885500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 853/1693] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:33 2022] POST /identity/v3/users => generated 299 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:33.886458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d76631-7f72-42bc-9ca7-9f7d609dffc2 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:33.910369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d76631-7f72-42bc-9ca7-9f7d609dffc2 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:33.928199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4477d1-20cc-454d-a04f-39587320ae63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:33.928826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4477d1-20cc-454d-a04f-39587320ae63 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:33.929481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4477d1-20cc-454d-a04f-39587320ae63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:33.929778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4477d1-20cc-454d-a04f-39587320ae63 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:33.930435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4477d1-20cc-454d-a04f-39587320ae63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:33.932056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4477d1-20cc-454d-a04f-39587320ae63 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:33.975444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 854/1694] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:33.990187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4477d1-20cc-454d-a04f-39587320ae63 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:34.011401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4477d1-20cc-454d-a04f-39587320ae63 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:34.035114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e03b-b7fd-459c-92e7-bbce7f5d929f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:34.035796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e03b-b7fd-459c-92e7-bbce7f5d929f admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.036716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e03b-b7fd-459c-92e7-bbce7f5d929f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.037565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646e03b-b7fd-459c-92e7-bbce7f5d929f admin admin] PATH_INFO: `/v3/projects/ed638c22842d490eade42fc9fc192fb7/users/9e844329570c4804aed6498051947646/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.048637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:34.049787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.050557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.051733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.082557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646e03b-b7fd-459c-92e7-bbce7f5d929f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed638c22842d490eade42fc9fc192fb7, user_id=9e844329570c4804aed6498051947646, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:34.087322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646e03b-b7fd-459c-92e7-bbce7f5d929f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:34.114769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 855/1695] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:33 2022] PUT /identity/v3/projects/ed638c22842d490eade42fc9fc192fb7/users/9e844329570c4804aed6498051947646/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:34.118897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e03b-b7fd-459c-92e7-bbce7f5d929f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:34.173541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:34.243362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646e03b-b7fd-459c-92e7-bbce7f5d929f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:34.261251 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02d720-1c0b-474f-b9eb-7122f252b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:34.261974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02d720-1c0b-474f-b9eb-7122f252b4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.262327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02d720-1c0b-474f-b9eb-7122f252b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.262671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02d720-1c0b-474f-b9eb-7122f252b4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.263400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02d720-1c0b-474f-b9eb-7122f252b4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:34.265377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02d720-1c0b-474f-b9eb-7122f252b4c5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:34.266246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:34.274809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:34.283634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 856/1696] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:34.283634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8593a1-0130-4759-9835-8f634b837b7a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.283634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8593a1-0130-4759-9835-8f634b837b7a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.283634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8593a1-0130-4759-9835-8f634b837b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.299024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 841/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:48:34.301786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:34.303422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:34.332941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-3d8593a1-0130-4759-9835-8f634b837b7a None None] MFA Rules not processed for user `3153c9ff011f4ab8958fc49795e6f8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:34.334743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3d8593a1-0130-4759-9835-8f634b837b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:34.360067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb87d9-b3f7-42e5-b923-ae5d9fc59e3e service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:34.403411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:34.503628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 857/1698] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:34.512751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8593a1-0130-4759-9835-8f634b837b7a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:34.512830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5c7c2-9f49-4d41-a119-e06dbb720d18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:34.512830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c7c2-9f49-4d41-a119-e06dbb720d18 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.512830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c7c2-9f49-4d41-a119-e06dbb720d18 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.515685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5c7c2-9f49-4d41-a119-e06dbb720d18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.519791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5cd5c7c2-9f49-4d41-a119-e06dbb720d18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:34.528816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8593a1-0130-4759-9835-8f634b837b7a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:34.541719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c8428-fe9d-4af7-a1ec-9db6b7865e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:34.543583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c8428-fe9d-4af7-a1ec-9db6b7865e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.543864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c8428-fe9d-4af7-a1ec-9db6b7865e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.544580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c8428-fe9d-4af7-a1ec-9db6b7865e3d admin admin] PATH_INFO: `/v3/projects/ed638c22842d490eade42fc9fc192fb7/users/9e844329570c4804aed6498051947646/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.560551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c8428-fe9d-4af7-a1ec-9db6b7865e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed638c22842d490eade42fc9fc192fb7, user_id=9e844329570c4804aed6498051947646, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:34.570162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c8428-fe9d-4af7-a1ec-9db6b7865e3d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:34.615210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 858/1699] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:34 2022] PUT /identity/v3/projects/ed638c22842d490eade42fc9fc192fb7/users/9e844329570c4804aed6498051947646/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:34.615210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb28605-45d0-4e1e-b274-c13a03f8ae8a None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.615210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb28605-45d0-4e1e-b274-c13a03f8ae8a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.615210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb28605-45d0-4e1e-b274-c13a03f8ae8a None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.615210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 859/1700] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 9 01:48:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:48:34.628039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f5945-5d3f-4406-862b-3df169500b7f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.628677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f5945-5d3f-4406-862b-3df169500b7f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.628965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f5945-5d3f-4406-862b-3df169500b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.647548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5c7c2-9f49-4d41-a119-e06dbb720d18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:34.653718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5c7c2-9f49-4d41-a119-e06dbb720d18 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:34.678345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 842/1701] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:34.679820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ca60-4da4-4baa-893b-26ab54d49a19 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.680111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ca60-4da4-4baa-893b-26ab54d49a19 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.680398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ca60-4da4-4baa-893b-26ab54d49a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.708396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-293f5945-5d3f-4406-862b-3df169500b7f None None] MFA Rules not processed for user `3153c9ff011f4ab8958fc49795e6f8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:34.714867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-293f5945-5d3f-4406-862b-3df169500b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:34.764441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-50b2ca60-4da4-4baa-893b-26ab54d49a19 None None] MFA Rules not processed for user `9e844329570c4804aed6498051947646`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:34.767915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-50b2ca60-4da4-4baa-893b-26ab54d49a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:34.835702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 860/1702] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:34.836911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5146d-e1a3-4248-b445-e7fcc9959b25 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:34.837234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5146d-e1a3-4248-b445-e7fcc9959b25 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:34.837512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5146d-e1a3-4248-b445-e7fcc9959b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:34.904967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f5d5146d-e1a3-4248-b445-e7fcc9959b25 None None] MFA Rules not processed for user `341277e7e3144e5ea93fa657916821e1`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:34.906488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f5d5146d-e1a3-4248-b445-e7fcc9959b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:34.954464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 843/1703] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:34.954464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2ca60-4da4-4baa-893b-26ab54d49a19 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:34.959192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-50b2ca60-4da4-4baa-893b-26ab54d49a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:34.992509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:35.011241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 861/1704] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:48:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:35.025561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5146d-e1a3-4248-b445-e7fcc9959b25 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:35.142462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:35.160362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5146d-e1a3-4248-b445-e7fcc9959b25 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:35.168166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:35.169333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:35.179442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:35.179834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:35.185811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2ca60-4da4-4baa-893b-26ab54d49a19 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:35.195258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:35.197766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:35.199951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7fe0a-ddd3-42d6-8b5c-478038ac82a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:35.200575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7fe0a-ddd3-42d6-8b5c-478038ac82a1 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:35.200867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7fe0a-ddd3-42d6-8b5c-478038ac82a1 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:35.203578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7fe0a-ddd3-42d6-8b5c-478038ac82a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:35.227355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 862/1705] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:35.228502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:35.232356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:35.235226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7fe0a-ddd3-42d6-8b5c-478038ac82a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:35.242499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7fe0a-ddd3-42d6-8b5c-478038ac82a1 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:35.273481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 844/1706] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:35.274533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7fe0a-ddd3-42d6-8b5c-478038ac82a1 service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:35.290253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c169b4-1b40-4aa9-ab1a-52ed5c5b50c9 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:35.307891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2024d-5501-4090-8fa4-cfcfe4a101fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:35.308410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2024d-5501-4090-8fa4-cfcfe4a101fe service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:35.308698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2024d-5501-4090-8fa4-cfcfe4a101fe service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:35.308974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2024d-5501-4090-8fa4-cfcfe4a101fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:35.337621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2024d-5501-4090-8fa4-cfcfe4a101fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:35.341021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2024d-5501-4090-8fa4-cfcfe4a101fe service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:35.363183 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 863/1707] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:35.421794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7fe0a-ddd3-42d6-8b5c-478038ac82a1 service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:35.433765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:35.437229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:35.438933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:35.440485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] PATH_INFO: `/v3/users/fe7be079e4754539874fe6a989d3ce28` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:35.446115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe7be079e4754539874fe6a989d3ce28)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:35.448859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:35.526841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe7be079e4754539874fe6a989d3ce28'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:35.528537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe7be079e4754539874fe6a989d3ce28'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:35.541576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe7be079e4754539874fe6a989d3ce28'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:35.564890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe7be079e4754539874fe6a989d3ce28'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:35.574391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe7be079e4754539874fe6a989d3ce28'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:35.581925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 845/1708] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:35 2022] DELETE /identity/v3/users/fe7be079e4754539874fe6a989d3ce28 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:35.586395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae61cc-2b45-4a51-b408-62d8456e28ec None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:35.586680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae61cc-2b45-4a51-b408-62d8456e28ec None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:35.586949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae61cc-2b45-4a51-b408-62d8456e28ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:35.608802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a4ae61cc-2b45-4a51-b408-62d8456e28ec None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:35.610227 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a4ae61cc-2b45-4a51-b408-62d8456e28ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:35.736026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 864/1709] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:35.779023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:35.785692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:35.838851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:35.954213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75ee63-dc8f-418a-995a-4008a9fa37df admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:35.971993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec6c41-52af-423e-b803-a82b41ac017f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:35.972637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6c41-52af-423e-b803-a82b41ac017f service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:35.972916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6c41-52af-423e-b803-a82b41ac017f service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:35.974897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec6c41-52af-423e-b803-a82b41ac017f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:36.005970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec6c41-52af-423e-b803-a82b41ac017f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:36.014819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec6c41-52af-423e-b803-a82b41ac017f service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:36.040492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 846/1710] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:36.057411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e160cf6-f46f-4630-add3-613967590ba7 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:36.057829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e160cf6-f46f-4630-add3-613967590ba7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:36.058133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e160cf6-f46f-4630-add3-613967590ba7 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:36.059887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 865/1711] 10.222.0.15 () {54 vars in 912 bytes} [Fri Sep 9 01:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:48:36.069410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7820-263f-424b-acdf-867894f1336a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:36.070066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7820-263f-424b-acdf-867894f1336a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:36.070545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf7820-263f-424b-acdf-867894f1336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:36.108106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-93bf7820-263f-424b-acdf-867894f1336a None None] MFA Rules not processed for user `341277e7e3144e5ea93fa657916821e1`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:36.109387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-93bf7820-263f-424b-acdf-867894f1336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:36.126798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 847/1712] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:48:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:36.158910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e160cf6-f46f-4630-add3-613967590ba7 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:36.179148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e160cf6-f46f-4630-add3-613967590ba7 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:36.191154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:36.191922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:36.192448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:36.193252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:36.218342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:36.221674 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:36.246604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 866/1713] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:48:36.287927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf33f-c7eb-4216-b93d-7bea8a9d73cb None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:36.288640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf33f-c7eb-4216-b93d-7bea8a9d73cb None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:36.289158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bf33f-c7eb-4216-b93d-7bea8a9d73cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:36.342011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-725bf33f-c7eb-4216-b93d-7bea8a9d73cb None None] MFA Rules not processed for user `83f9dc8c41604f0e86eaf57202285cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:36.343466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-725bf33f-c7eb-4216-b93d-7bea8a9d73cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:36.500786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 848/1714] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:36.516314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:36.518812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:36.654753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc1cf1-a5cd-4d97-9d33-b64aaf682c3a service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:36.660677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:36.661999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:36.662402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:36.662673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:36.665324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:36.696083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:36.701555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:36.725274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 867/1715] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:36.855608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf33f-c7eb-4216-b93d-7bea8a9d73cb None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:36.863779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-725bf33f-c7eb-4216-b93d-7bea8a9d73cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:36.882730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:36.884208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:36.908964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5fdda-978b-4cc6-b328-185ceb3eb0d9 service placement] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:36.920269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:36.920707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:36.920986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:36.921321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] PATH_INFO: `/v3/projects/6be3300e9b3d4c30ab20b8995187c499` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:36.924149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be3300e9b3d4c30ab20b8995187c499)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:36.927171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:37.030124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be3300e9b3d4c30ab20b8995187c499', 'request_id': 'req-829716ea-40bc-4ab8-8823-5f16400d4e79'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.031512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:37.040789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be3300e9b3d4c30ab20b8995187c499', 'request_id': 'req-829716ea-40bc-4ab8-8823-5f16400d4e79'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.049004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be3300e9b3d4c30ab20b8995187c499', 'request_id': 'req-829716ea-40bc-4ab8-8823-5f16400d4e79'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.069937 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] The token cache is being invalidate because project 6be3300e9b3d4c30ab20b8995187c499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:37.070340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.078475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.083983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 868/1716] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:36 2022] DELETE /identity/v3/projects/6be3300e9b3d4c30ab20b8995187c499 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:37.084154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bf33f-c7eb-4216-b93d-7bea8a9d73cb None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:37.085438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:37.087344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:37.175021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:37.197799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829716ea-40bc-4ab8-8823-5f16400d4e79 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:37.205733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:37.206415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:37.206945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:37.207237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] PATH_INFO: `/v3/users/9e844329570c4804aed6498051947646` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:37.210094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e844329570c4804aed6498051947646)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:37.212939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:37.238599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:37.244685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:37.245114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:37.245496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:37.248671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:37.289686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e844329570c4804aed6498051947646'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.290436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e844329570c4804aed6498051947646'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.331721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e844329570c4804aed6498051947646'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.350663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e844329570c4804aed6498051947646'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.352035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-5213287d-9e31-4973-8c62-bf4060fae47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e844329570c4804aed6498051947646'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:37.418450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 869/1717] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:37 2022] DELETE /identity/v3/users/9e844329570c4804aed6498051947646 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:37.430925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:37.432845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:37.499687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 849/1718] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:37.515237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a5825-0906-46aa-a5e4-d590fbcd3849 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:37.515764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a5825-0906-46aa-a5e4-d590fbcd3849 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:37.516265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a5825-0906-46aa-a5e4-d590fbcd3849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:37.574400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4f8a5825-0906-46aa-a5e4-d590fbcd3849 None None] MFA Rules not processed for user `3153c9ff011f4ab8958fc49795e6f8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:37.575710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4f8a5825-0906-46aa-a5e4-d590fbcd3849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:37.756934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 870/1719] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:38.456933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:38.485964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15376fd3-5f67-4e26-a9ac-65f1083a6af2 service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:38.743397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:38.744361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:38.744800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:38.745354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] PATH_INFO: `/v3/projects/ed638c22842d490eade42fc9fc192fb7` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:38.749768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed638c22842d490eade42fc9fc192fb7)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:38.757563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:38.919438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed638c22842d490eade42fc9fc192fb7', 'request_id': 'req-82573cdb-5a18-400a-a174-554582a1f16a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:38.940855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed638c22842d490eade42fc9fc192fb7', 'request_id': 'req-82573cdb-5a18-400a-a174-554582a1f16a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:38.965661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed638c22842d490eade42fc9fc192fb7', 'request_id': 'req-82573cdb-5a18-400a-a174-554582a1f16a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:39.013643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] The token cache is being invalidate because project ed638c22842d490eade42fc9fc192fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:39.013935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:39.014834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-82573cdb-5a18-400a-a174-554582a1f16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:39.016724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 850/1720] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:38 2022] DELETE /identity/v3/projects/ed638c22842d490eade42fc9fc192fb7 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:39.022005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252a6ac-d691-4ef7-b20f-43259f608772 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.022856 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252a6ac-d691-4ef7-b20f-43259f608772 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.023866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252a6ac-d691-4ef7-b20f-43259f608772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:39.056505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:39.084346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-0252a6ac-d691-4ef7-b20f-43259f608772 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:39.090365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0252a6ac-d691-4ef7-b20f-43259f608772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:39.253094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 871/1721] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:39.262902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02affc-ea96-49b4-9962-004f5e8e80a8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.263209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02affc-ea96-49b4-9962-004f5e8e80a8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.263470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02affc-ea96-49b4-9962-004f5e8e80a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:39.311656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:39.331125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1b02affc-ea96-49b4-9962-004f5e8e80a8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:39.340296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1b02affc-ea96-49b4-9962-004f5e8e80a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:39.351898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 851/1722] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:39.355784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252a6ac-d691-4ef7-b20f-43259f608772 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:39.373311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252a6ac-d691-4ef7-b20f-43259f608772 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:39.387327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:39.388050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.388401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.388732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:39.389484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:39.389892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:39.391587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:39.428076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 872/1723] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:39.432685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02affc-ea96-49b4-9962-004f5e8e80a8 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:39.446970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02affc-ea96-49b4-9962-004f5e8e80a8 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:39.463363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267699bf-73ca-4570-8732-e366e7e450bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:39.464002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267699bf-73ca-4570-8732-e366e7e450bb admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.464286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267699bf-73ca-4570-8732-e366e7e450bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.464560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267699bf-73ca-4570-8732-e366e7e450bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:39.466831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267699bf-73ca-4570-8732-e366e7e450bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:39.472552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267699bf-73ca-4570-8732-e366e7e450bb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:39.548856 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 852/1724] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:39 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:39.550486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:39.568035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f1257-2aed-4675-a958-a9f699b7a6b7 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:39.700037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee5e68-85b9-47e9-ac5f-66f7680f8064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:39.701107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee5e68-85b9-47e9-ac5f-66f7680f8064 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.701107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee5e68-85b9-47e9-ac5f-66f7680f8064 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.701449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee5e68-85b9-47e9-ac5f-66f7680f8064 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:39.702096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee5e68-85b9-47e9-ac5f-66f7680f8064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:39.704041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee5e68-85b9-47e9-ac5f-66f7680f8064 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:39.758322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 873/1725] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:39 2022] POST /identity/v3/users => generated 294 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:39.767310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267699bf-73ca-4570-8732-e366e7e450bb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:39.793977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267699bf-73ca-4570-8732-e366e7e450bb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:39.803291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2015c-42ce-40a0-b241-7289b364e4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:39.804475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2015c-42ce-40a0-b241-7289b364e4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.804766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2015c-42ce-40a0-b241-7289b364e4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.805040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2015c-42ce-40a0-b241-7289b364e4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:39.805871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2015c-42ce-40a0-b241-7289b364e4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:39.807491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2015c-42ce-40a0-b241-7289b364e4e0 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:39.821515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 853/1726] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:39.825601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee5e68-85b9-47e9-ac5f-66f7680f8064 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:39.838592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee5e68-85b9-47e9-ac5f-66f7680f8064 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:39.857311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebe506-53ad-41df-9078-c2f92f280f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:39.859024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebe506-53ad-41df-9078-c2f92f280f02 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.860415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebe506-53ad-41df-9078-c2f92f280f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.861514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebe506-53ad-41df-9078-c2f92f280f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:39.862414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebe506-53ad-41df-9078-c2f92f280f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:39.866708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebe506-53ad-41df-9078-c2f92f280f02 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:39.881835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 874/1727] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:39.886331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2015c-42ce-40a0-b241-7289b364e4e0 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:39.899827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2015c-42ce-40a0-b241-7289b364e4e0 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:39.909182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c5ae1-5704-46a4-8e15-164639427c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:39.911545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c5ae1-5704-46a4-8e15-164639427c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.911903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c5ae1-5704-46a4-8e15-164639427c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.912249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c5ae1-5704-46a4-8e15-164639427c9d admin admin] PATH_INFO: `/v3/projects/cb75ea0bc23c4525ae10d4f815794e07/users/245583d367eb401a961eb2c796fad70e/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:39.937525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c5ae1-5704-46a4-8e15-164639427c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb75ea0bc23c4525ae10d4f815794e07, user_id=245583d367eb401a961eb2c796fad70e, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:39.940622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c5ae1-5704-46a4-8e15-164639427c9d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:39.961980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 854/1728] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:39 2022] PUT /identity/v3/projects/cb75ea0bc23c4525ae10d4f815794e07/users/245583d367eb401a961eb2c796fad70e/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:39.973166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69d4b5-af0a-4868-aa20-93cf6ef713d4 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:39.973461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69d4b5-af0a-4868-aa20-93cf6ef713d4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:39.973735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69d4b5-af0a-4868-aa20-93cf6ef713d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:40.024957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5d69d4b5-af0a-4868-aa20-93cf6ef713d4 None None] MFA Rules not processed for user `245583d367eb401a961eb2c796fad70e`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:40.026815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5d69d4b5-af0a-4868-aa20-93cf6ef713d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:40.170706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:40.196331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 875/1729] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:40.197637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c5ae1-5704-46a4-8e15-164639427c9d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:40.337976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c5ae1-5704-46a4-8e15-164639427c9d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:40.356440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189e1c9-c46c-4340-8223-e1f91e3cbceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:40.357132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189e1c9-c46c-4340-8223-e1f91e3cbceb admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:40.357423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189e1c9-c46c-4340-8223-e1f91e3cbceb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:40.357698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189e1c9-c46c-4340-8223-e1f91e3cbceb admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:40.358339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189e1c9-c46c-4340-8223-e1f91e3cbceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:40.360258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e189e1c9-c46c-4340-8223-e1f91e3cbceb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:40.433922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 855/1730] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:40 2022] POST /identity/v3/projects => generated 383 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:40.438220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69d4b5-af0a-4868-aa20-93cf6ef713d4 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:40.450081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69d4b5-af0a-4868-aa20-93cf6ef713d4 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:40.530528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646623b-63ca-4733-8f24-6e116e14d629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:40.531307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646623b-63ca-4733-8f24-6e116e14d629 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:40.531756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646623b-63ca-4733-8f24-6e116e14d629 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:40.531949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646623b-63ca-4733-8f24-6e116e14d629 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:40.532742 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646623b-63ca-4733-8f24-6e116e14d629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:40.534822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d646623b-63ca-4733-8f24-6e116e14d629 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:40.562809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 876/1731] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:40 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:40.564015 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189e1c9-c46c-4340-8223-e1f91e3cbceb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:40.575090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189e1c9-c46c-4340-8223-e1f91e3cbceb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:40.581130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d497c-90f0-4c9a-b3a6-2edf31419152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:40.583916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d497c-90f0-4c9a-b3a6-2edf31419152 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:40.584430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d497c-90f0-4c9a-b3a6-2edf31419152 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:40.584915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d497c-90f0-4c9a-b3a6-2edf31419152 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:40.585663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d497c-90f0-4c9a-b3a6-2edf31419152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:40.587462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d497c-90f0-4c9a-b3a6-2edf31419152 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:40.609751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646623b-63ca-4733-8f24-6e116e14d629 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:40.623303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 856/1732] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:40.629878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d646623b-63ca-4733-8f24-6e116e14d629 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:40.635915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b391f-0484-441b-ada2-2d64ef51dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:40.636606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b391f-0484-441b-ada2-2d64ef51dcfe admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:40.636890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b391f-0484-441b-ada2-2d64ef51dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:40.637253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b391f-0484-441b-ada2-2d64ef51dcfe admin admin] PATH_INFO: `/v3/projects/42bc5852513b4b9484844bd089dddb28/users/2759c1bdcdfd4a81b405a879931d0d7c/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:40.664277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b391f-0484-441b-ada2-2d64ef51dcfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bc5852513b4b9484844bd089dddb28, user_id=2759c1bdcdfd4a81b405a879931d0d7c, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:40.667004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b391f-0484-441b-ada2-2d64ef51dcfe admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:40.691469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 877/1733] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:40 2022] PUT /identity/v3/projects/42bc5852513b4b9484844bd089dddb28/users/2759c1bdcdfd4a81b405a879931d0d7c/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:40.695809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d497c-90f0-4c9a-b3a6-2edf31419152 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:40.715122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d497c-90f0-4c9a-b3a6-2edf31419152 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:40.802482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1693ff-ee6b-415f-8863-cc3446328327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:40.803218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1693ff-ee6b-415f-8863-cc3446328327 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:40.803599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1693ff-ee6b-415f-8863-cc3446328327 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:40.803987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1693ff-ee6b-415f-8863-cc3446328327 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:40.804735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1693ff-ee6b-415f-8863-cc3446328327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:40.806929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1693ff-ee6b-415f-8863-cc3446328327 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:40.821135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 857/1734] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:40.824278 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b391f-0484-441b-ada2-2d64ef51dcfe admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:40.837102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b391f-0484-441b-ada2-2d64ef51dcfe admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:40.841724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:40.842406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:40.843490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:40.843490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] PATH_INFO: `/v3/projects/42bc5852513b4b9484844bd089dddb28/users/2759c1bdcdfd4a81b405a879931d0d7c/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:40.847294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bc5852513b4b9484844bd089dddb28, user_id=2759c1bdcdfd4a81b405a879931d0d7c, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:40.850601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:40.878607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e265fb8-055d-418f-8487-737184f0aae8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:40.879176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e265fb8-055d-418f-8487-737184f0aae8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:40.882223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e265fb8-055d-418f-8487-737184f0aae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:40.888004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 878/1735] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:40 2022] PUT /identity/v3/projects/42bc5852513b4b9484844bd089dddb28/users/2759c1bdcdfd4a81b405a879931d0d7c/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:40.929393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:40.934339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4e265fb8-055d-418f-8487-737184f0aae8 None None] MFA Rules not processed for user `2759c1bdcdfd4a81b405a879931d0d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:40.936638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4e265fb8-055d-418f-8487-737184f0aae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:41.047489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 858/1736] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:41.061374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:41.064211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:41.131534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2e67c7-e050-47d9-8383-c2ebe63d9e80 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:41.135627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e251e-fa5a-4b04-9f82-ca1f9bd16072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:41.136712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e251e-fa5a-4b04-9f82-ca1f9bd16072 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:41.136984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e251e-fa5a-4b04-9f82-ca1f9bd16072 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:41.137285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e251e-fa5a-4b04-9f82-ca1f9bd16072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:41.147905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e251e-fa5a-4b04-9f82-ca1f9bd16072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:41.149365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e251e-fa5a-4b04-9f82-ca1f9bd16072 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:41.159287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 879/1737] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:41.238540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e265fb8-055d-418f-8487-737184f0aae8 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:41.264915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d9db3-fa75-4a59-bd36-63abd96f509c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:41.265224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d9db3-fa75-4a59-bd36-63abd96f509c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:41.265501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d9db3-fa75-4a59-bd36-63abd96f509c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:41.287030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-061d9db3-fa75-4a59-bd36-63abd96f509c None None] MFA Rules not processed for user `83f9dc8c41604f0e86eaf57202285cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:41.288310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-061d9db3-fa75-4a59-bd36-63abd96f509c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:41.310208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e265fb8-055d-418f-8487-737184f0aae8 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:41.315049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:41.315686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:41.315960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:41.316261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] PATH_INFO: `/v3/users/245583d367eb401a961eb2c796fad70e` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:41.317962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] RBAC: Authorizing `identity:delete_user(user_id=245583d367eb401a961eb2c796fad70e)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:41.319866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:41.363234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:41.372833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245583d367eb401a961eb2c796fad70e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:41.373824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245583d367eb401a961eb2c796fad70e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:41.376436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 880/1738] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:41.383586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245583d367eb401a961eb2c796fad70e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:41.398259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245583d367eb401a961eb2c796fad70e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:41.406307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245583d367eb401a961eb2c796fad70e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:41.415455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 859/1739] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:41 2022] DELETE /identity/v3/users/245583d367eb401a961eb2c796fad70e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:41.420666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc6198-18e7-4652-ada7-5cdc2e484867 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:41.420949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc6198-18e7-4652-ada7-5cdc2e484867 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:41.421239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc6198-18e7-4652-ada7-5cdc2e484867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:41.471482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1fcc6198-18e7-4652-ada7-5cdc2e484867 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:41.478203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1fcc6198-18e7-4652-ada7-5cdc2e484867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:41.611809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 881/1740] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:41.621980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:41.623653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:41.665578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:41.712691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72b0ee-b6fd-4fb5-bdeb-8db993cbecca admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:41.719237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d6b5e-dde2-4e64-af54-49ecd25bd651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:41.719880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6b5e-dde2-4e64-af54-49ecd25bd651 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:41.720151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6b5e-dde2-4e64-af54-49ecd25bd651 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:41.720524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d6b5e-dde2-4e64-af54-49ecd25bd651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:41.747756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d6b5e-dde2-4e64-af54-49ecd25bd651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:41.750133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d6b5e-dde2-4e64-af54-49ecd25bd651 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:41.765774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 860/1741] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:42.393501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc6198-18e7-4652-ada7-5cdc2e484867 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:42.408844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1fcc6198-18e7-4652-ada7-5cdc2e484867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:42.439161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc6198-18e7-4652-ada7-5cdc2e484867 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:42.471824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:42.472980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:42.474550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:42.474885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] PATH_INFO: `/v3/projects/cb75ea0bc23c4525ae10d4f815794e07` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:42.477652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb75ea0bc23c4525ae10d4f815794e07)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:42.483890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:42.622865 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb75ea0bc23c4525ae10d4f815794e07', 'request_id': 'req-2f10ab26-1f02-40ee-83aa-f4211831e1e2'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.648371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb75ea0bc23c4525ae10d4f815794e07', 'request_id': 'req-2f10ab26-1f02-40ee-83aa-f4211831e1e2'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.666256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb75ea0bc23c4525ae10d4f815794e07', 'request_id': 'req-2f10ab26-1f02-40ee-83aa-f4211831e1e2'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.684672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] The token cache is being invalidate because project cb75ea0bc23c4525ae10d4f815794e07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:42.685169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.685830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.687775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 882/1742] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:42 2022] DELETE /identity/v3/projects/cb75ea0bc23c4525ae10d4f815794e07 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:42.692409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d6b5e-dde2-4e64-af54-49ecd25bd651 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:42.704969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d6b5e-dde2-4e64-af54-49ecd25bd651 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:42.780812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:42.781512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:42.781795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:42.782070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] PATH_INFO: `/v3/users/2759c1bdcdfd4a81b405a879931d0d7c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:42.783215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2759c1bdcdfd4a81b405a879931d0d7c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:42.785256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:42.868549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2759c1bdcdfd4a81b405a879931d0d7c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.869151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2759c1bdcdfd4a81b405a879931d0d7c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.885085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2759c1bdcdfd4a81b405a879931d0d7c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.903922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2759c1bdcdfd4a81b405a879931d0d7c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.919252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-636c451f-de14-47d6-a2a9-294b0e6cbdd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2759c1bdcdfd4a81b405a879931d0d7c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:42.936056 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 861/1743] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:42 2022] DELETE /identity/v3/users/2759c1bdcdfd4a81b405a879931d0d7c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:43.381956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:43.383453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:43.410556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10ab26-1f02-40ee-83aa-f4211831e1e2 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:43.546557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:43.548258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:43.548399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:43.551799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] PATH_INFO: `/v3/projects/42bc5852513b4b9484844bd089dddb28` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:43.564172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42bc5852513b4b9484844bd089dddb28)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:43.566421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:43.645446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:43.686972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42bc5852513b4b9484844bd089dddb28', 'request_id': 'req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:43.695622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42bc5852513b4b9484844bd089dddb28', 'request_id': 'req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:43.706596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42bc5852513b4b9484844bd089dddb28', 'request_id': 'req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:43.726461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] The token cache is being invalidate because project 42bc5852513b4b9484844bd089dddb28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:43.726917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:43.730679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0b52dc01-1fc7-46cd-b3e1-11dd2a1fc082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:43.739777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 883/1744] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:43 2022] DELETE /identity/v3/projects/42bc5852513b4b9484844bd089dddb28 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:43.741342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ac6e-d340-4a85-a0d7-94f6d02b9831 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:43.741835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ac6e-d340-4a85-a0d7-94f6d02b9831 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:43.741929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853ac6e-d340-4a85-a0d7-94f6d02b9831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:43.805456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8853ac6e-d340-4a85-a0d7-94f6d02b9831 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:43.806745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8853ac6e-d340-4a85-a0d7-94f6d02b9831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:43.860580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:43.906388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 862/1745] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:43.915663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a18c59-633e-415f-97eb-1a09f1fcf036 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:43.916067 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a18c59-633e-415f-97eb-1a09f1fcf036 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:43.916485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a18c59-633e-415f-97eb-1a09f1fcf036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:43.940041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-69a18c59-633e-415f-97eb-1a09f1fcf036 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:43.941294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-69a18c59-633e-415f-97eb-1a09f1fcf036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:43.950715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 884/1746] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:43.954988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853ac6e-d340-4a85-a0d7-94f6d02b9831 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:43.972286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853ac6e-d340-4a85-a0d7-94f6d02b9831 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:43.992589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:43.993761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:43.994198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:43.994725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:43.996048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:43.996456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:44.000010 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:44.042475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 863/1747] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:44.047684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a18c59-633e-415f-97eb-1a09f1fcf036 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:44.066018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a18c59-633e-415f-97eb-1a09f1fcf036 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:44.094033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbbca4-ae62-4983-ad2a-9cf2260757a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:44.096327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbbca4-ae62-4983-ad2a-9cf2260757a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:44.096832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbbca4-ae62-4983-ad2a-9cf2260757a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:44.097419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbbca4-ae62-4983-ad2a-9cf2260757a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:44.098939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbbca4-ae62-4983-ad2a-9cf2260757a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:44.101166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbbca4-ae62-4983-ad2a-9cf2260757a3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:44.145530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 885/1748] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:44 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:44.155604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:44.172527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f3a77-2886-451c-a1da-7e7119da896e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:44.249055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b1554-2d07-40ef-a11c-65a242fe392e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:44.249702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1554-2d07-40ef-a11c-65a242fe392e admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:44.249976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1554-2d07-40ef-a11c-65a242fe392e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:44.250247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1554-2d07-40ef-a11c-65a242fe392e admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:44.250878 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b1554-2d07-40ef-a11c-65a242fe392e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:44.252783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b1554-2d07-40ef-a11c-65a242fe392e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:44.295449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 864/1749] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:44 2022] POST /identity/v3/users => generated 301 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:44.301103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbbca4-ae62-4983-ad2a-9cf2260757a3 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:44.312238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbbca4-ae62-4983-ad2a-9cf2260757a3 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:44.318081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c770a9f-09df-4d08-af95-10e24de6814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:44.318722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c770a9f-09df-4d08-af95-10e24de6814f admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:44.319062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c770a9f-09df-4d08-af95-10e24de6814f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:44.319454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c770a9f-09df-4d08-af95-10e24de6814f admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:44.320135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c770a9f-09df-4d08-af95-10e24de6814f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:44.323051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c770a9f-09df-4d08-af95-10e24de6814f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:44.336024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 886/1750] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:44.339801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b1554-2d07-40ef-a11c-65a242fe392e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:44.356880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b1554-2d07-40ef-a11c-65a242fe392e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:44.373775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e398fa4-0ffa-4708-9380-8c8f94411acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:44.374239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e398fa4-0ffa-4708-9380-8c8f94411acd admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:44.374538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e398fa4-0ffa-4708-9380-8c8f94411acd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:44.374784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e398fa4-0ffa-4708-9380-8c8f94411acd admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:44.375820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e398fa4-0ffa-4708-9380-8c8f94411acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:44.382393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e398fa4-0ffa-4708-9380-8c8f94411acd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:44.416724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c770a9f-09df-4d08-af95-10e24de6814f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:44.418286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 865/1751] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:48:44.434485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c770a9f-09df-4d08-af95-10e24de6814f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:44.466843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d799674-68b8-4615-be49-56b8f21390dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:44.466843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d799674-68b8-4615-be49-56b8f21390dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:44.466843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d799674-68b8-4615-be49-56b8f21390dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:44.466843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d799674-68b8-4615-be49-56b8f21390dc admin admin] PATH_INFO: `/v3/projects/f1862262c5964f08899d9863f683fe79/users/2e2255b716504dff9d747e8380052935/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:44.493808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d799674-68b8-4615-be49-56b8f21390dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1862262c5964f08899d9863f683fe79, user_id=2e2255b716504dff9d747e8380052935, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:44.499369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d799674-68b8-4615-be49-56b8f21390dc admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:44.556019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 887/1752] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:44 2022] PUT /identity/v3/projects/f1862262c5964f08899d9863f683fe79/users/2e2255b716504dff9d747e8380052935/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:44.561330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96940f-3dd5-4da9-8a8a-4b08101c8efa None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:44.561330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96940f-3dd5-4da9-8a8a-4b08101c8efa None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:44.561330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96940f-3dd5-4da9-8a8a-4b08101c8efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:44.578986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d55df-ff7e-4744-b416-f3ec1373a91b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:44.579270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d55df-ff7e-4744-b416-f3ec1373a91b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:44.579537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d55df-ff7e-4744-b416-f3ec1373a91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:44.620749 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9a96940f-3dd5-4da9-8a8a-4b08101c8efa None None] MFA Rules not processed for user `2e2255b716504dff9d747e8380052935`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:44.621957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9a96940f-3dd5-4da9-8a8a-4b08101c8efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:44.691555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b36d55df-ff7e-4744-b416-f3ec1373a91b None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:44.693561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b36d55df-ff7e-4744-b416-f3ec1373a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:44.846769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 866/1753] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:44 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:44.851589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96940f-3dd5-4da9-8a8a-4b08101c8efa None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:44.941119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 888/1754] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:48:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:48:44.982827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d55df-ff7e-4744-b416-f3ec1373a91b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:44.985600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:44.986680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b36d55df-ff7e-4744-b416-f3ec1373a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:45.055788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:45.080398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96940f-3dd5-4da9-8a8a-4b08101c8efa None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:45.087608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:45.088644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:45.089006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:45.089519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:45.091443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:45.103119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:45.177740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 867/1755] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:44 2022] POST /identity/v3/projects => generated 397 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:48:45.177740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:45.179183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:45.179901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d55df-ff7e-4744-b416-f3ec1373a91b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:45.378748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:45.379926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:45.380208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:45.380483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] PATH_INFO: `/v3/users/98e198600f974feaaa0e62a23d690f25` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:45.391307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e198600f974feaaa0e62a23d690f25)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:45.396405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:45.539019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e6c9-5776-4afe-b9cf-2c1a19744b61 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:45.554767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e81e3-af0e-4a75-a462-f3229d77179e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:45.555715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e81e3-af0e-4a75-a462-f3229d77179e service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:45.557100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e81e3-af0e-4a75-a462-f3229d77179e service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:45.557631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e81e3-af0e-4a75-a462-f3229d77179e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:45.587753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e198600f974feaaa0e62a23d690f25'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:45.590876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e198600f974feaaa0e62a23d690f25'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:45.626261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e198600f974feaaa0e62a23d690f25'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:45.641244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e198600f974feaaa0e62a23d690f25'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:45.641922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e198600f974feaaa0e62a23d690f25'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:45.664858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 889/1756] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:44 2022] DELETE /identity/v3/users/98e198600f974feaaa0e62a23d690f25 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:45.666590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:45.678974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:45.771547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e81e3-af0e-4a75-a462-f3229d77179e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:45.774398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e81e3-af0e-4a75-a462-f3229d77179e service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:45.826233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 868/1757] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:48:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:48:45.827555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58116d-d3b0-462a-b7c3-b4264baead20 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:45.863249 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58116d-d3b0-462a-b7c3-b4264baead20 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:45.864432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58116d-d3b0-462a-b7c3-b4264baead20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:45.911783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-2b58116d-d3b0-462a-b7c3-b4264baead20 None None] MFA Rules not processed for user `341277e7e3144e5ea93fa657916821e1`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:45.913330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2b58116d-d3b0-462a-b7c3-b4264baead20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:45.963940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:45.990074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:46.002087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc6ee4b-e8cc-42ee-8f0b-980b432c02fd admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.011805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81c58a-1880-4969-892e-33253ee2c1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.012467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81c58a-1880-4969-892e-33253ee2c1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.012771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81c58a-1880-4969-892e-33253ee2c1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.013032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81c58a-1880-4969-892e-33253ee2c1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.013706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81c58a-1880-4969-892e-33253ee2c1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.015712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81c58a-1880-4969-892e-33253ee2c1a1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.023981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 869/1758] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Sep 9 01:48:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:46.025526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05232cc3-1f2f-445d-b6ea-aacdf350ec1e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.025922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05232cc3-1f2f-445d-b6ea-aacdf350ec1e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.026269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05232cc3-1f2f-445d-b6ea-aacdf350ec1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.064486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 890/1759] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:45 2022] POST /identity/v3/users => generated 305 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:46.065788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81c58a-1880-4969-892e-33253ee2c1a1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.076250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-05232cc3-1f2f-445d-b6ea-aacdf350ec1e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:46.077649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-05232cc3-1f2f-445d-b6ea-aacdf350ec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.085242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81c58a-1880-4969-892e-33253ee2c1a1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.088342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 870/1760] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:46.095486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145d784-0e6f-40ae-9117-9466fa409408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.096219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145d784-0e6f-40ae-9117-9466fa409408 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.096571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145d784-0e6f-40ae-9117-9466fa409408 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.096917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145d784-0e6f-40ae-9117-9466fa409408 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.097704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145d784-0e6f-40ae-9117-9466fa409408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.099440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145d784-0e6f-40ae-9117-9466fa409408 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.104762 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05232cc3-1f2f-445d-b6ea-aacdf350ec1e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.106930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-05232cc3-1f2f-445d-b6ea-aacdf350ec1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.118201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 891/1761] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:46.122126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145d784-0e6f-40ae-9117-9466fa409408 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.124042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05232cc3-1f2f-445d-b6ea-aacdf350ec1e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.128177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd72f40-7453-4e13-a34d-c38acb82f634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.128812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd72f40-7453-4e13-a34d-c38acb82f634 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.129114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd72f40-7453-4e13-a34d-c38acb82f634 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.129922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd72f40-7453-4e13-a34d-c38acb82f634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.137693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145d784-0e6f-40ae-9117-9466fa409408 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.142911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd72f40-7453-4e13-a34d-c38acb82f634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.143186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fef5ba-fe9e-47f5-9e6b-d96bbee95d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.144136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fef5ba-fe9e-47f5-9e6b-d96bbee95d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.144467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd72f40-7453-4e13-a34d-c38acb82f634 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.144592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fef5ba-fe9e-47f5-9e6b-d96bbee95d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.144923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fef5ba-fe9e-47f5-9e6b-d96bbee95d74 admin admin] PATH_INFO: `/v3/projects/6880dee6035948febfb316e7320e7b81/users/3c50cb03f508462ba136bbf76ce895b6/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.164958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 871/1762] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:46.179040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fef5ba-fe9e-47f5-9e6b-d96bbee95d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6880dee6035948febfb316e7320e7b81, user_id=3c50cb03f508462ba136bbf76ce895b6, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.183060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fef5ba-fe9e-47f5-9e6b-d96bbee95d74 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.203579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 892/1763] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:46 2022] PUT /identity/v3/projects/6880dee6035948febfb316e7320e7b81/users/3c50cb03f508462ba136bbf76ce895b6/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:46.207813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd72f40-7453-4e13-a34d-c38acb82f634 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.219406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd72f40-7453-4e13-a34d-c38acb82f634 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.337897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8727370-d08d-4396-ae24-142c8f3cfb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.338726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727370-d08d-4396-ae24-142c8f3cfb4f admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.339443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727370-d08d-4396-ae24-142c8f3cfb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.339812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727370-d08d-4396-ae24-142c8f3cfb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.340637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8727370-d08d-4396-ae24-142c8f3cfb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.349381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8727370-d08d-4396-ae24-142c8f3cfb4f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.369695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 872/1764] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:46.371023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fef5ba-fe9e-47f5-9e6b-d96bbee95d74 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.399075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fef5ba-fe9e-47f5-9e6b-d96bbee95d74 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.422582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.423269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.423556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.424098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] PATH_INFO: `/v3/projects/6880dee6035948febfb316e7320e7b81/users/3c50cb03f508462ba136bbf76ce895b6/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.426919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6880dee6035948febfb316e7320e7b81, user_id=3c50cb03f508462ba136bbf76ce895b6, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.429903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.453105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 893/1765] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:46 2022] PUT /identity/v3/projects/6880dee6035948febfb316e7320e7b81/users/3c50cb03f508462ba136bbf76ce895b6/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:46.464324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c377c40-5e04-4d79-8603-e6a1265a791e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.465209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c377c40-5e04-4d79-8603-e6a1265a791e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.465941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c377c40-5e04-4d79-8603-e6a1265a791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.542037 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-2c377c40-5e04-4d79-8603-e6a1265a791e None None] MFA Rules not processed for user `3c50cb03f508462ba136bbf76ce895b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:46.543674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2c377c40-5e04-4d79-8603-e6a1265a791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.565616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.568049 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.670213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 873/1766] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:46 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:46.677435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c377c40-5e04-4d79-8603-e6a1265a791e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.679615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2c377c40-5e04-4d79-8603-e6a1265a791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.692335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfc662-e608-4935-85ae-8b2bce317864 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.698169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.698668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.698942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.699235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] PATH_INFO: `/v3/users/c8f3463f4c534fa7b2a87e5cac6c3a70` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.700554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f3463f4c534fa7b2a87e5cac6c3a70)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.702520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.705647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c377c40-5e04-4d79-8603-e6a1265a791e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.711788 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.712452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.712730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.712998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] PATH_INFO: `/v3/projects/6d01372b1bdb420fbdbda8d44285fbbe` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.714145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d01372b1bdb420fbdbda8d44285fbbe)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.716049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.762249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f3463f4c534fa7b2a87e5cac6c3a70'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.763218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f3463f4c534fa7b2a87e5cac6c3a70'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.771215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d01372b1bdb420fbdbda8d44285fbbe', 'request_id': 'req-cb2091b7-8be4-48cf-8fbd-91434b757e30'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.773147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:46.778561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f3463f4c534fa7b2a87e5cac6c3a70'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.779734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d01372b1bdb420fbdbda8d44285fbbe', 'request_id': 'req-cb2091b7-8be4-48cf-8fbd-91434b757e30'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.788401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f3463f4c534fa7b2a87e5cac6c3a70'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.789042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f3463f4c534fa7b2a87e5cac6c3a70'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.789640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d01372b1bdb420fbdbda8d44285fbbe', 'request_id': 'req-cb2091b7-8be4-48cf-8fbd-91434b757e30'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.804306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 894/1767] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:46 2022] DELETE /identity/v3/users/c8f3463f4c534fa7b2a87e5cac6c3a70 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:46.805392 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.806738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.810618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] The token cache is being invalidate because project 6d01372b1bdb420fbdbda8d44285fbbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:46.810983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.811704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cb2091b7-8be4-48cf-8fbd-91434b757e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:46.813240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 874/1768] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:46 2022] DELETE /identity/v3/projects/6d01372b1bdb420fbdbda8d44285fbbe => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:46.814403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1101871-3f76-4fc5-867c-910d650939a4 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.814714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1101871-3f76-4fc5-867c-910d650939a4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.814991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1101871-3f76-4fc5-867c-910d650939a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.820249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:46.834309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f1101871-3f76-4fc5-867c-910d650939a4 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:46.835497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f1101871-3f76-4fc5-867c-910d650939a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.874667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903e09d-5a53-45ab-bd99-86c9d92d3a74 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:46.880031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff027a66-8be7-4bb6-ab54-2c036848b84e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:46.880787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027a66-8be7-4bb6-ab54-2c036848b84e service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.881105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027a66-8be7-4bb6-ab54-2c036848b84e service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.881366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff027a66-8be7-4bb6-ab54-2c036848b84e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.883123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ff027a66-8be7-4bb6-ab54-2c036848b84e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.904196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 875/1769] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:46.905638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd7377-bced-4f92-adb8-42c4cc8d3ad1 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.905928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd7377-bced-4f92-adb8-42c4cc8d3ad1 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.906204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd7377-bced-4f92-adb8-42c4cc8d3ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:46.928814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-3bcd7377-bced-4f92-adb8-42c4cc8d3ad1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:46.932522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3bcd7377-bced-4f92-adb8-42c4cc8d3ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.943487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 876/1770] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:46.944572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd7377-bced-4f92-adb8-42c4cc8d3ad1 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:46.946690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3bcd7377-bced-4f92-adb8-42c4cc8d3ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:46.980840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff027a66-8be7-4bb6-ab54-2c036848b84e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:46.986956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff027a66-8be7-4bb6-ab54-2c036848b84e service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:46.993477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 895/1771] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:46 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:46.995439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bde49-7bf2-4088-b208-2d2d72524324 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:46.995726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bde49-7bf2-4088-b208-2d2d72524324 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:46.996001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bde49-7bf2-4088-b208-2d2d72524324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.018019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f55bde49-7bf2-4088-b208-2d2d72524324 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:47.019503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f55bde49-7bf2-4088-b208-2d2d72524324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:47.033092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 896/1772] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:47.037094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bde49-7bf2-4088-b208-2d2d72524324 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.046318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd7377-bced-4f92-adb8-42c4cc8d3ad1 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.047647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bde49-7bf2-4088-b208-2d2d72524324 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.050158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.050928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.051315 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.051690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.053231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.053898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.054321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.054603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.055473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:47.055803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.057489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.063986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.065719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.073108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 897/1773] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:47.076460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.082295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 877/1774] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:47.086050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.087352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:47.088140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25795d0b-78d7-40a0-8453-004c5d94c404 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.094645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445216f-ae57-4d75-8090-16afba0968ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.096106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445216f-ae57-4d75-8090-16afba0968ca admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.096408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445216f-ae57-4d75-8090-16afba0968ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.096689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445216f-ae57-4d75-8090-16afba0968ca admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.097656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445216f-ae57-4d75-8090-16afba0968ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.099323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa02b1-ad55-4f57-9c38-2983a9500a1b service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.101389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445216f-ae57-4d75-8090-16afba0968ca admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.103003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.103592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.103990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.104266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.106708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:47.148384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 898/1775] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:47.153163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445216f-ae57-4d75-8090-16afba0968ca admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.183190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.184657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.197374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 878/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:47 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:47.229730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445216f-ae57-4d75-8090-16afba0968ca admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.234668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e692a61-36b6-407c-8232-a4f615df5fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.235279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e692a61-36b6-407c-8232-a4f615df5fbb admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.235656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e692a61-36b6-407c-8232-a4f615df5fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.235938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e692a61-36b6-407c-8232-a4f615df5fbb admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.236854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e692a61-36b6-407c-8232-a4f615df5fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.239861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e692a61-36b6-407c-8232-a4f615df5fbb admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.271320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 899/1777] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:48:47.272848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.285882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80311b6-3991-4464-ac7a-76d89a06024d service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.290760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7830-5ef8-4b73-a29a-5a962aa05ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.291496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7830-5ef8-4b73-a29a-5a962aa05ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.291965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7830-5ef8-4b73-a29a-5a962aa05ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.292493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7830-5ef8-4b73-a29a-5a962aa05ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.293460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7830-5ef8-4b73-a29a-5a962aa05ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.295631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7830-5ef8-4b73-a29a-5a962aa05ae4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.305754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:47.316672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 879/1778] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:47.316936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e692a61-36b6-407c-8232-a4f615df5fbb admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.323891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:47.330045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e692a61-36b6-407c-8232-a4f615df5fbb admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.338398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.339830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.340333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.340806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.341953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.343877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.360862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 900/1779] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:48:47.361986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7830-5ef8-4b73-a29a-5a962aa05ae4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.383916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7830-5ef8-4b73-a29a-5a962aa05ae4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.391169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478443a-d0f8-4e60-9bf9-0c61c104340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.391892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478443a-d0f8-4e60-9bf9-0c61c104340c admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.392129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478443a-d0f8-4e60-9bf9-0c61c104340c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.392479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478443a-d0f8-4e60-9bf9-0c61c104340c admin admin] PATH_INFO: `/v3/projects/4a96185e97084d11ade97d9fda7b1c57/users/97d46fe05322492c85d0ea30c1d0b8e4/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.404744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.406792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:47.428927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478443a-d0f8-4e60-9bf9-0c61c104340c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a96185e97084d11ade97d9fda7b1c57, user_id=97d46fe05322492c85d0ea30c1d0b8e4, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.428927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478443a-d0f8-4e60-9bf9-0c61c104340c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.437335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9a27c-283c-4cc7-b092-b018b8e35575 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.453758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 880/1780] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:47 2022] PUT /identity/v3/projects/4a96185e97084d11ade97d9fda7b1c57/users/97d46fe05322492c85d0ea30c1d0b8e4/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:47.467701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4e10-ed85-4d77-808c-7f23a3aae75c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.468344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4e10-ed85-4d77-808c-7f23a3aae75c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.468810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c4e10-ed85-4d77-808c-7f23a3aae75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.510428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.511222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.511569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.511918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] PATH_INFO: `/v3/projects/daa9c339892144d281422ad480383a22` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.513439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6e3c4e10-ed85-4d77-808c-7f23a3aae75c None None] MFA Rules not processed for user `97d46fe05322492c85d0ea30c1d0b8e4`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:47.535163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa9c339892144d281422ad480383a22)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.535163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.541804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6e3c4e10-ed85-4d77-808c-7f23a3aae75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:47.643048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa9c339892144d281422ad480383a22', 'request_id': 'req-0e01ae53-4eaa-4e75-a19b-06f61396fa97'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:47.651718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa9c339892144d281422ad480383a22', 'request_id': 'req-0e01ae53-4eaa-4e75-a19b-06f61396fa97'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:47.667448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa9c339892144d281422ad480383a22', 'request_id': 'req-0e01ae53-4eaa-4e75-a19b-06f61396fa97'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:47.676964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 881/1781] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:47.677930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c4e10-ed85-4d77-808c-7f23a3aae75c None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.690096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c4e10-ed85-4d77-808c-7f23a3aae75c None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.694859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] The token cache is being invalidate because project daa9c339892144d281422ad480383a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:47.695327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:47.696090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0e01ae53-4eaa-4e75-a19b-06f61396fa97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:47.701455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 901/1782] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:47 2022] DELETE /identity/v3/projects/daa9c339892144d281422ad480383a22 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:48:47.703039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f8df-0c11-43dc-952c-4efa78950e10 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.703578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f8df-0c11-43dc-952c-4efa78950e10 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.704003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0f8df-0c11-43dc-952c-4efa78950e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.748163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-8fa0f8df-0c11-43dc-952c-4efa78950e10 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:47.749466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8fa0f8df-0c11-43dc-952c-4efa78950e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:47.762231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.762840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.763118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.763391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.764013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.765986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.797035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 882/1783] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:47.798239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.800222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:47.809073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 902/1784] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:47.817193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fbd27-d552-4606-8f10-43f23b818e8a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.817473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fbd27-d552-4606-8f10-43f23b818e8a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.817748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650fbd27-d552-4606-8f10-43f23b818e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.837341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-650fbd27-d552-4606-8f10-43f23b818e8a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:47.838455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-650fbd27-d552-4606-8f10-43f23b818e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:47.851190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:47.857098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:47.866304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a254a-1c33-4383-a7bc-a282dd243cf5 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.871414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeff83d-4b09-4242-a2a3-0512f8331b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.872023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeff83d-4b09-4242-a2a3-0512f8331b99 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.872298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeff83d-4b09-4242-a2a3-0512f8331b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.872568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeff83d-4b09-4242-a2a3-0512f8331b99 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.873506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeff83d-4b09-4242-a2a3-0512f8331b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.875417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeff83d-4b09-4242-a2a3-0512f8331b99 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.900497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 883/1785] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:47.901253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 903/1786] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:48:47.901622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeff83d-4b09-4242-a2a3-0512f8331b99 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.904121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fbd27-d552-4606-8f10-43f23b818e8a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.912930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeff83d-4b09-4242-a2a3-0512f8331b99 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.913427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650fbd27-d552-4606-8f10-43f23b818e8a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.918343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54c8ef-9d7a-4d31-8877-c4e09e921d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.918978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c8ef-9d7a-4d31-8877-c4e09e921d5d admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.919260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c8ef-9d7a-4d31-8877-c4e09e921d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.919533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54c8ef-9d7a-4d31-8877-c4e09e921d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.920177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54c8ef-9d7a-4d31-8877-c4e09e921d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.920788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.921581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.921828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54c8ef-9d7a-4d31-8877-c4e09e921d5d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.922166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.922584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.923227 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:47.923525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.925128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.936423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 884/1787] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:48:47.939577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 904/1788] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:47.942214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.942353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54c8ef-9d7a-4d31-8877-c4e09e921d5d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.951811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54c8ef-9d7a-4d31-8877-c4e09e921d5d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.952021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c0fb1-05ba-4bf7-a6d7-6393f29cb6ef admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:47.956371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b026eef-9b8e-4af5-8e7c-9d6aaefddaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.956637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4588e-4b4b-48a8-9beb-0c1511543823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:47.957029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026eef-9b8e-4af5-8e7c-9d6aaefddaf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.957318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4588e-4b4b-48a8-9beb-0c1511543823 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:47.957407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026eef-9b8e-4af5-8e7c-9d6aaefddaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.957586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b026eef-9b8e-4af5-8e7c-9d6aaefddaf3 admin admin] PATH_INFO: `/v3/projects/f1c4504e74094ecc96ff658c92646e01/users/4c7d906aa64242b78c67638aa62621da/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.957677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4588e-4b4b-48a8-9beb-0c1511543823 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:47.957861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4588e-4b4b-48a8-9beb-0c1511543823 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:47.958502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4588e-4b4b-48a8-9beb-0c1511543823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.961445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4588e-4b4b-48a8-9beb-0c1511543823 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.972043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b026eef-9b8e-4af5-8e7c-9d6aaefddaf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c4504e74094ecc96ff658c92646e01, user_id=4c7d906aa64242b78c67638aa62621da, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:47.975485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b026eef-9b8e-4af5-8e7c-9d6aaefddaf3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:47.993971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 905/1789] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:48:47.994943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4588e-4b4b-48a8-9beb-0c1511543823 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:47.998417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 885/1790] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:47 2022] PUT /identity/v3/projects/f1c4504e74094ecc96ff658c92646e01/users/4c7d906aa64242b78c67638aa62621da/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:48:48.001926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b026eef-9b8e-4af5-8e7c-9d6aaefddaf3 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:48.060080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4588e-4b4b-48a8-9beb-0c1511543823 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:48.064901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be783c25-7bfd-426d-b979-305656287b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:48.065637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783c25-7bfd-426d-b979-305656287b70 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.065912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783c25-7bfd-426d-b979-305656287b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.066271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be783c25-7bfd-426d-b979-305656287b70 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.067091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b026eef-9b8e-4af5-8e7c-9d6aaefddaf3 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:48.067473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be783c25-7bfd-426d-b979-305656287b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:48.069432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be783c25-7bfd-426d-b979-305656287b70 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:48.074067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c94e-27f2-41d8-a3cd-474b153fbba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:48.074651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c94e-27f2-41d8-a3cd-474b153fbba7 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.074936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c94e-27f2-41d8-a3cd-474b153fbba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.075206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258c94e-27f2-41d8-a3cd-474b153fbba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.075839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258c94e-27f2-41d8-a3cd-474b153fbba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:48.077745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258c94e-27f2-41d8-a3cd-474b153fbba7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:48.093295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 886/1791] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:48:48.093444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 906/1792] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:47 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:48:48.095480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be783c25-7bfd-426d-b979-305656287b70 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:48.095790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c94e-27f2-41d8-a3cd-474b153fbba7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:48.105042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be783c25-7bfd-426d-b979-305656287b70 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:48.105688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258c94e-27f2-41d8-a3cd-474b153fbba7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:48.109620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8371d4-48f9-4b62-bba3-a159abc064c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:48.110254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8371d4-48f9-4b62-bba3-a159abc064c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.110355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eeb9e-7f75-4f41-aceb-e562098cf502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:48.110548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8371d4-48f9-4b62-bba3-a159abc064c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.110787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8371d4-48f9-4b62-bba3-a159abc064c5 admin admin] PATH_INFO: `/v3/projects/f1c4504e74094ecc96ff658c92646e01/users/4c7d906aa64242b78c67638aa62621da/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.110896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eeb9e-7f75-4f41-aceb-e562098cf502 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.111113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eeb9e-7f75-4f41-aceb-e562098cf502 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.111377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eeb9e-7f75-4f41-aceb-e562098cf502 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.112030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959eeb9e-7f75-4f41-aceb-e562098cf502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:48.112661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8371d4-48f9-4b62-bba3-a159abc064c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c4504e74094ecc96ff658c92646e01, user_id=4c7d906aa64242b78c67638aa62621da, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:48.114122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959eeb9e-7f75-4f41-aceb-e562098cf502 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:48.115811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8371d4-48f9-4b62-bba3-a159abc064c5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:48.124556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 887/1793] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:48:48.126546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 907/1794] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:48 2022] PUT /identity/v3/projects/f1c4504e74094ecc96ff658c92646e01/users/4c7d906aa64242b78c67638aa62621da/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:48.133989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eeb9e-7f75-4f41-aceb-e562098cf502 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:48.137136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7efa2c-14f5-4cee-9eb3-5ba60156390e None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.137346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7efa2c-14f5-4cee-9eb3-5ba60156390e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.137569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7efa2c-14f5-4cee-9eb3-5ba60156390e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.144186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eeb9e-7f75-4f41-aceb-e562098cf502 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:48.165040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2b7efa2c-14f5-4cee-9eb3-5ba60156390e None None] MFA Rules not processed for user `4c7d906aa64242b78c67638aa62621da`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:48.166281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2b7efa2c-14f5-4cee-9eb3-5ba60156390e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:48.202204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67b186-562f-43ad-ba8c-50a59389891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:48.202811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b186-562f-43ad-ba8c-50a59389891d admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.203084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b186-562f-43ad-ba8c-50a59389891d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.203353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67b186-562f-43ad-ba8c-50a59389891d admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.203990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67b186-562f-43ad-ba8c-50a59389891d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:48.205594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67b186-562f-43ad-ba8c-50a59389891d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:48.215318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 888/1795] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:48.219527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67b186-562f-43ad-ba8c-50a59389891d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:48.222993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:48.229427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67b186-562f-43ad-ba8c-50a59389891d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:48.235097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9f83a-bac3-4e7e-a2be-81b9fd65b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:48.235789 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9f83a-bac3-4e7e-a2be-81b9fd65b783 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.236064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9f83a-bac3-4e7e-a2be-81b9fd65b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.236329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9f83a-bac3-4e7e-a2be-81b9fd65b783 admin admin] PATH_INFO: `/v3/projects/c5a7dae40be34e94ab73a9358bddd3e2/users/0df3f44d2ce74312810ec8886a37c97a/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.243240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 908/1796] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:48:48.253695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9f83a-bac3-4e7e-a2be-81b9fd65b783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a7dae40be34e94ab73a9358bddd3e2, user_id=0df3f44d2ce74312810ec8886a37c97a, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:48.254147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7efa2c-14f5-4cee-9eb3-5ba60156390e None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:48.255561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2b7efa2c-14f5-4cee-9eb3-5ba60156390e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:48.256781 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9f83a-bac3-4e7e-a2be-81b9fd65b783 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:48.283612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 889/1797] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:48 2022] PUT /identity/v3/projects/c5a7dae40be34e94ab73a9358bddd3e2/users/0df3f44d2ce74312810ec8886a37c97a/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:48:48.286518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335283c-5b26-475c-9618-5f721a9bfa15 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.286853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335283c-5b26-475c-9618-5f721a9bfa15 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.287133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335283c-5b26-475c-9618-5f721a9bfa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.297798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:48.337954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6335283c-5b26-475c-9618-5f721a9bfa15 None None] MFA Rules not processed for user `0df3f44d2ce74312810ec8886a37c97a`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:48.339198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6335283c-5b26-475c-9618-5f721a9bfa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:48.356034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7efa2c-14f5-4cee-9eb3-5ba60156390e None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:48.441961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ff752-4b9f-4303-bb1f-fc8d5d500fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:48.443039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ff752-4b9f-4303-bb1f-fc8d5d500fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.443523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ff752-4b9f-4303-bb1f-fc8d5d500fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.444255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ff752-4b9f-4303-bb1f-fc8d5d500fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.448346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 890/1798] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:48:48.460908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335283c-5b26-475c-9618-5f721a9bfa15 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:48.462363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6335283c-5b26-475c-9618-5f721a9bfa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:48.485575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335283c-5b26-475c-9618-5f721a9bfa15 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:48.497573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb89a1-1f70-4d41-b8c5-47fb33e106b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:48.498801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb89a1-1f70-4d41-b8c5-47fb33e106b4 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:48.499787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb89a1-1f70-4d41-b8c5-47fb33e106b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:48.500065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb89a1-1f70-4d41-b8c5-47fb33e106b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:48.510337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb89a1-1f70-4d41-b8c5-47fb33e106b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:48.511993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb89a1-1f70-4d41-b8c5-47fb33e106b4 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:48.522142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ff752-4b9f-4303-bb1f-fc8d5d500fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:48.523692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ff752-4b9f-4303-bb1f-fc8d5d500fd4 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:48.535041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 909/1799] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:48:48.553299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 891/1800] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:48:49.284421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ff752-4b9f-4303-bb1f-fc8d5d500fd4 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:49.342887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:49.347530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ff752-4b9f-4303-bb1f-fc8d5d500fd4 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:49.352635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:49.353313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:49.353589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:49.353858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] PATH_INFO: `/v3/users/97d46fe05322492c85d0ea30c1d0b8e4` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:49.354956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=97d46fe05322492c85d0ea30c1d0b8e4)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:49.356875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:49.402467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97d46fe05322492c85d0ea30c1d0b8e4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:49.403053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97d46fe05322492c85d0ea30c1d0b8e4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:49.415299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97d46fe05322492c85d0ea30c1d0b8e4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:49.423235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97d46fe05322492c85d0ea30c1d0b8e4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:49.423883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97d46fe05322492c85d0ea30c1d0b8e4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:49.435996 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 910/1801] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:49 2022] DELETE /identity/v3/users/97d46fe05322492c85d0ea30c1d0b8e4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:49.440617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf116b-2942-4a43-8d33-90d1299e7025 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:49.441182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf116b-2942-4a43-8d33-90d1299e7025 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:49.441339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf116b-2942-4a43-8d33-90d1299e7025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:49.464506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-c5cf116b-2942-4a43-8d33-90d1299e7025 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:49.465849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c5cf116b-2942-4a43-8d33-90d1299e7025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:49.526019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 892/1802] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:49.535522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:49.536909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:49.600117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42585a2-5448-4fab-bbfc-b2cd4468be6a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:49.603780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf211e-c049-4dd7-8670-c463fc001e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:49.604395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf211e-c049-4dd7-8670-c463fc001e6e service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:49.604678 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf211e-c049-4dd7-8670-c463fc001e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:49.604953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf211e-c049-4dd7-8670-c463fc001e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:49.614613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf211e-c049-4dd7-8670-c463fc001e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:49.616309 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf211e-c049-4dd7-8670-c463fc001e6e service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:49.626591 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 911/1803] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:48:50.048658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf116b-2942-4a43-8d33-90d1299e7025 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:50.050684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c5cf116b-2942-4a43-8d33-90d1299e7025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:50.064659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf116b-2942-4a43-8d33-90d1299e7025 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:50.070310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742413db-d810-436f-a921-9184539ada40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:50.070962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742413db-d810-436f-a921-9184539ada40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:50.071240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742413db-d810-436f-a921-9184539ada40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:50.071515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742413db-d810-436f-a921-9184539ada40 admin admin] PATH_INFO: `/v3/projects/4a96185e97084d11ade97d9fda7b1c57` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:50.072651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742413db-d810-436f-a921-9184539ada40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a96185e97084d11ade97d9fda7b1c57)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:50.074767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742413db-d810-436f-a921-9184539ada40 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:50.128992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-742413db-d810-436f-a921-9184539ada40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a96185e97084d11ade97d9fda7b1c57', 'request_id': 'req-742413db-d810-436f-a921-9184539ada40'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.135276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-742413db-d810-436f-a921-9184539ada40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a96185e97084d11ade97d9fda7b1c57', 'request_id': 'req-742413db-d810-436f-a921-9184539ada40'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.144481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-742413db-d810-436f-a921-9184539ada40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a96185e97084d11ade97d9fda7b1c57', 'request_id': 'req-742413db-d810-436f-a921-9184539ada40'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.160823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-742413db-d810-436f-a921-9184539ada40 admin admin] The token cache is being invalidate because project 4a96185e97084d11ade97d9fda7b1c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:50.161158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-742413db-d810-436f-a921-9184539ada40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.161827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-742413db-d810-436f-a921-9184539ada40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.166518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 893/1804] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:50 2022] DELETE /identity/v3/projects/4a96185e97084d11ade97d9fda7b1c57 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:50.167505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf211e-c049-4dd7-8670-c463fc001e6e service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:50.180597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf211e-c049-4dd7-8670-c463fc001e6e service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:50.236873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:50.237747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:50.238030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:50.238364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] PATH_INFO: `/v3/users/4c7d906aa64242b78c67638aa62621da` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:50.240027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7d906aa64242b78c67638aa62621da)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:50.242142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:50.264447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:50.292576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7d906aa64242b78c67638aa62621da'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.293270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7d906aa64242b78c67638aa62621da'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.306155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7d906aa64242b78c67638aa62621da'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.313299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7d906aa64242b78c67638aa62621da'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.313913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12acbb81-0cd4-4134-8455-bd652f999865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7d906aa64242b78c67638aa62621da'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.326036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 912/1805] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:48:50 2022] DELETE /identity/v3/users/4c7d906aa64242b78c67638aa62621da => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:48:50.643306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742413db-d810-436f-a921-9184539ada40 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:50.645844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-742413db-d810-436f-a921-9184539ada40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:50.655662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:50.661442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742413db-d810-436f-a921-9184539ada40 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:50.730611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:50.731263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:50.731541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:50.731841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] PATH_INFO: `/v3/projects/f1c4504e74094ecc96ff658c92646e01` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:50.733201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c4504e74094ecc96ff658c92646e01)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:50.735177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:50.796620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c4504e74094ecc96ff658c92646e01', 'request_id': 'req-91e31d6f-0607-4961-b3a7-9340215124f8'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.813254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c4504e74094ecc96ff658c92646e01', 'request_id': 'req-91e31d6f-0607-4961-b3a7-9340215124f8'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.820975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c4504e74094ecc96ff658c92646e01', 'request_id': 'req-91e31d6f-0607-4961-b3a7-9340215124f8'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.855417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] The token cache is being invalidate because project f1c4504e74094ecc96ff658c92646e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:48:50.855722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.861000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-91e31d6f-0607-4961-b3a7-9340215124f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:48:50.864296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 894/1806] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:48:50 2022] DELETE /identity/v3/projects/f1c4504e74094ecc96ff658c92646e01 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:50.870856 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8ce9-bed6-4221-920b-485bc18b7d3f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:50.871255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8ce9-bed6-4221-920b-485bc18b7d3f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:50.871531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e8ce9-bed6-4221-920b-485bc18b7d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:50.903471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-835e8ce9-bed6-4221-920b-485bc18b7d3f None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:50.903471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-835e8ce9-bed6-4221-920b-485bc18b7d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:50.965956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 913/1807] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:50.973802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33489d-33f8-4a56-bc0a-a3ed966658a0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:50.974097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33489d-33f8-4a56-bc0a-a3ed966658a0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:50.974453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33489d-33f8-4a56-bc0a-a3ed966658a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.007017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-bb33489d-33f8-4a56-bc0a-a3ed966658a0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:51.008231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bb33489d-33f8-4a56-bc0a-a3ed966658a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:51.016478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 895/1808] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:51.021283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e8ce9-bed6-4221-920b-485bc18b7d3f None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:51.030998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e8ce9-bed6-4221-920b-485bc18b7d3f None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:51.036237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:51.036844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:51.037155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:51.037426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.038065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:48:51.038372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:51.040161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:51.052185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 914/1809] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:48:51.057360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33489d-33f8-4a56-bc0a-a3ed966658a0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:51.067649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33489d-33f8-4a56-bc0a-a3ed966658a0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:51.077663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bd87f-50b4-4757-bedd-f03af7d72323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:51.078382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bd87f-50b4-4757-bedd-f03af7d72323 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:51.078735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bd87f-50b4-4757-bedd-f03af7d72323 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:51.079086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bd87f-50b4-4757-bedd-f03af7d72323 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.079814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bd87f-50b4-4757-bedd-f03af7d72323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:51.081830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521bd87f-50b4-4757-bedd-f03af7d72323 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:51.114539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 896/1810] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:48:51 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:51.115897 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:51.128104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc1db2-8853-4415-a87b-ec2b4d6b175b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:51.203731 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4c3cf-235f-4954-8f59-026a9dd6c2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:51.204397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4c3cf-235f-4954-8f59-026a9dd6c2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:51.204672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4c3cf-235f-4954-8f59-026a9dd6c2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:51.204942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4c3cf-235f-4954-8f59-026a9dd6c2e3 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.205682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4c3cf-235f-4954-8f59-026a9dd6c2e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:51.212684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4c3cf-235f-4954-8f59-026a9dd6c2e3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:51.237413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 915/1811] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:48:51 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:48:51.243186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bd87f-50b4-4757-bedd-f03af7d72323 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:51.256451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bd87f-50b4-4757-bedd-f03af7d72323 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:51.262840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001054d1-3285-48b2-8665-f3b721940ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:51.263544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001054d1-3285-48b2-8665-f3b721940ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:51.263899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001054d1-3285-48b2-8665-f3b721940ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:51.264355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001054d1-3285-48b2-8665-f3b721940ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.265152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001054d1-3285-48b2-8665-f3b721940ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:51.266883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001054d1-3285-48b2-8665-f3b721940ba4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:51.290345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 897/1812] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:48:51.294240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4c3cf-235f-4954-8f59-026a9dd6c2e3 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:51.307411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4c3cf-235f-4954-8f59-026a9dd6c2e3 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:51.312735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a2ec4-a2ad-470f-9ea6-a345c4a09d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:51.313393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a2ec4-a2ad-470f-9ea6-a345c4a09d18 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:51.313673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a2ec4-a2ad-470f-9ea6-a345c4a09d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:51.314016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a2ec4-a2ad-470f-9ea6-a345c4a09d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.314615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a2ec4-a2ad-470f-9ea6-a345c4a09d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:51.317040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a2ec4-a2ad-470f-9ea6-a345c4a09d18 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:51.329452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 916/1813] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:48:51.334253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001054d1-3285-48b2-8665-f3b721940ba4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:51.344440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001054d1-3285-48b2-8665-f3b721940ba4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:51.349713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:51.350421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:51.350698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:51.350988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] PATH_INFO: `/v3/projects/58c6ea5863b143e688246dde52e891d2/users/d99cc43956eb4a709472f39aaf76f488/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.371133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c6ea5863b143e688246dde52e891d2, user_id=d99cc43956eb4a709472f39aaf76f488, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:51.376675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:51.394143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 898/1814] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:48:51 2022] PUT /identity/v3/projects/58c6ea5863b143e688246dde52e891d2/users/d99cc43956eb4a709472f39aaf76f488/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:48:51.404746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa337fd5-182c-4fc4-a9e9-faaf36016b6d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:51.404994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa337fd5-182c-4fc4-a9e9-faaf36016b6d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:51.405321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa337fd5-182c-4fc4-a9e9-faaf36016b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.414006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:51.456610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-fa337fd5-182c-4fc4-a9e9-faaf36016b6d None None] MFA Rules not processed for user `d99cc43956eb4a709472f39aaf76f488`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:51.458786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-fa337fd5-182c-4fc4-a9e9-faaf36016b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:51.539993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 917/1815] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:48:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:51.551144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:51.552853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:51.590049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:48:51.648860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3683-9d33-4ac0-a957-a3c1b2a9439f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:51.653037 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029447e782484e8f8a704b683ede1819', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:51.653666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:51.653964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:51.654230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:51.663065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:51.664489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:51.673758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 899/1816] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:48:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:48:57.720008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29499e8-b45b-4162-a207-b2b3b790217f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:57.720541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29499e8-b45b-4162-a207-b2b3b790217f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:57.720677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29499e8-b45b-4162-a207-b2b3b790217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:57.743115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e29499e8-b45b-4162-a207-b2b3b790217f None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:48:57.744397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e29499e8-b45b-4162-a207-b2b3b790217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:57.800688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 918/1817] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:48:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:48:57.812733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:48:57.814288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:48:57.902581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3dc3d-dfc1-4378-958c-14f29dec901b service glance] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:48:57.907908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22594b0-62fe-469f-9f4f-affee1b01c18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:48:57.908568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22594b0-62fe-469f-9f4f-affee1b01c18 service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:48:57.908789 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22594b0-62fe-469f-9f4f-affee1b01c18 service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:48:57.909084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22594b0-62fe-469f-9f4f-affee1b01c18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:48:57.919131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22594b0-62fe-469f-9f4f-affee1b01c18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:48:57.920867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22594b0-62fe-469f-9f4f-affee1b01c18 service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:48:57.931538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 900/1818] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:48:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:49:00.547224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29499e8-b45b-4162-a207-b2b3b790217f None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:00.548843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e29499e8-b45b-4162-a207-b2b3b790217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:00.633468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29499e8-b45b-4162-a207-b2b3b790217f None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:00.638015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:00.638650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:00.638930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:00.639235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] PATH_INFO: `/v3/users/df3d135051b14af78ddb93494a2119c0` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:00.640375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df3d135051b14af78ddb93494a2119c0)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:00.642325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:00.688498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:00.695237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df3d135051b14af78ddb93494a2119c0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:00.696079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df3d135051b14af78ddb93494a2119c0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:00.709437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df3d135051b14af78ddb93494a2119c0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:00.718469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df3d135051b14af78ddb93494a2119c0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:00.719154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df3d135051b14af78ddb93494a2119c0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:00.733232 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 919/1819] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:00 2022] DELETE /identity/v3/users/df3d135051b14af78ddb93494a2119c0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:00.746672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070e204-67e0-4235-a1fb-5a5ebccfbacc None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:00.747996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070e204-67e0-4235-a1fb-5a5ebccfbacc None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:00.748270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070e204-67e0-4235-a1fb-5a5ebccfbacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:00.773758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6070e204-67e0-4235-a1fb-5a5ebccfbacc None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:00.774719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6070e204-67e0-4235-a1fb-5a5ebccfbacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:00.883709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 901/1820] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:00.889725 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:00.891799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:01.031822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a34803-9989-4761-8c26-224326664e23 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:01.044169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809516d-8ef7-4016-ac25-fde24c8757c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:01.045018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809516d-8ef7-4016-ac25-fde24c8757c4 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:01.045281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809516d-8ef7-4016-ac25-fde24c8757c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:01.045562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809516d-8ef7-4016-ac25-fde24c8757c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:01.056540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809516d-8ef7-4016-ac25-fde24c8757c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:01.060552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809516d-8ef7-4016-ac25-fde24c8757c4 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:01.072064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 920/1821] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:01.426617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070e204-67e0-4235-a1fb-5a5ebccfbacc None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:01.428299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6070e204-67e0-4235-a1fb-5a5ebccfbacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:01.443451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070e204-67e0-4235-a1fb-5a5ebccfbacc None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:01.449701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:01.461636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:01.461897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:01.462359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] PATH_INFO: `/v3/projects/88a0a47943c74230b45151abca2271e5` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:01.464283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a0a47943c74230b45151abca2271e5)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:01.466584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:01.510294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:01.520320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a0a47943c74230b45151abca2271e5', 'request_id': 'req-81dcde29-b71f-4f87-87d6-e02557aa9d00'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.526498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a0a47943c74230b45151abca2271e5', 'request_id': 'req-81dcde29-b71f-4f87-87d6-e02557aa9d00'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.532909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a0a47943c74230b45151abca2271e5', 'request_id': 'req-81dcde29-b71f-4f87-87d6-e02557aa9d00'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.544839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] The token cache is being invalidate because project 88a0a47943c74230b45151abca2271e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:01.545162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.545776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.547243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 902/1822] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:01 2022] DELETE /identity/v3/projects/88a0a47943c74230b45151abca2271e5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:01.551269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809516d-8ef7-4016-ac25-fde24c8757c4 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:01.561375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809516d-8ef7-4016-ac25-fde24c8757c4 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:01.617246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:01.617878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:01.618156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:01.618429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] PATH_INFO: `/v3/users/5918220422994787ac84f9681a391c0a` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:01.619547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5918220422994787ac84f9681a391c0a)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:01.621487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:01.668648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5918220422994787ac84f9681a391c0a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.669292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5918220422994787ac84f9681a391c0a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.680585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:01.682022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5918220422994787ac84f9681a391c0a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.689518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5918220422994787ac84f9681a391c0a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.690469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-970896c7-2de7-43a9-97eb-4c8ee4703e78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5918220422994787ac84f9681a391c0a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:01.703214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 921/1823] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:01 2022] DELETE /identity/v3/users/5918220422994787ac84f9681a391c0a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:01.979016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:01.980792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:01.996377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcde29-b71f-4f87-87d6-e02557aa9d00 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.065176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.065176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.065176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.065176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] PATH_INFO: `/v3/projects/591cdb1b657a41c5b9f2cb500fbfd58d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.066176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=591cdb1b657a41c5b9f2cb500fbfd58d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.068113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.119636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591cdb1b657a41c5b9f2cb500fbfd58d', 'request_id': 'req-47a55982-eef9-4210-8a9d-0e7f726bec50'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.126554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591cdb1b657a41c5b9f2cb500fbfd58d', 'request_id': 'req-47a55982-eef9-4210-8a9d-0e7f726bec50'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.133153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591cdb1b657a41c5b9f2cb500fbfd58d', 'request_id': 'req-47a55982-eef9-4210-8a9d-0e7f726bec50'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.146895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] The token cache is being invalidate because project 591cdb1b657a41c5b9f2cb500fbfd58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:02.147187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.147800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47a55982-eef9-4210-8a9d-0e7f726bec50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.149532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 903/1824] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:01 2022] DELETE /identity/v3/projects/591cdb1b657a41c5b9f2cb500fbfd58d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:02.154257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b669b-2717-45eb-a167-492cfb8e330a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.154540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b669b-2717-45eb-a167-492cfb8e330a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.154810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b669b-2717-45eb-a167-492cfb8e330a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.174875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-044b669b-2717-45eb-a167-492cfb8e330a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:02.176022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-044b669b-2717-45eb-a167-492cfb8e330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:02.239655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 922/1825] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:02.247428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203143ed-244b-4457-b3d8-577dd378e9d3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.247970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203143ed-244b-4457-b3d8-577dd378e9d3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.248243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203143ed-244b-4457-b3d8-577dd378e9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.274931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-203143ed-244b-4457-b3d8-577dd378e9d3 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:02.276676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-203143ed-244b-4457-b3d8-577dd378e9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:02.296728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 904/1826] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:02.302330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b669b-2717-45eb-a167-492cfb8e330a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.318403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b669b-2717-45eb-a167-492cfb8e330a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.323689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.324318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.327730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.327730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.327730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:02.327730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.328170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.328701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203143ed-244b-4457-b3d8-577dd378e9d3 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.330114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-203143ed-244b-4457-b3d8-577dd378e9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:02.341562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 923/1827] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:02.343053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203143ed-244b-4457-b3d8-577dd378e9d3 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.345679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.347859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.349194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.349578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.350099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] PATH_INFO: `/v3/users/2e2255b716504dff9d747e8380052935` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.353755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2255b716504dff9d747e8380052935)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.356351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef004d-08ab-4d2d-beb0-931e1457ef65 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.360500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.362323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61f81d-4c6b-493f-8245-8161bd7e115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.363376 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61f81d-4c6b-493f-8245-8161bd7e115c admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.364069 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61f81d-4c6b-493f-8245-8161bd7e115c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.364436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a61f81d-4c6b-493f-8245-8161bd7e115c admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.365233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61f81d-4c6b-493f-8245-8161bd7e115c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.367328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a61f81d-4c6b-493f-8245-8161bd7e115c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.412380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 924/1828] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:02 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:02.413439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61f81d-4c6b-493f-8245-8161bd7e115c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.450401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:02.454239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2255b716504dff9d747e8380052935'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.455573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2255b716504dff9d747e8380052935'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.466479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2255b716504dff9d747e8380052935'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.482440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2255b716504dff9d747e8380052935'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.493016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3531413f-d179-49c9-b7d0-ac271a841901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2255b716504dff9d747e8380052935'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:02.499933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 905/1829] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:02 2022] DELETE /identity/v3/users/2e2255b716504dff9d747e8380052935 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:02.504204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb17bc-9d02-48b3-906a-36ed15d8e748 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.504674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb17bc-9d02-48b3-906a-36ed15d8e748 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.505052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb17bc-9d02-48b3-906a-36ed15d8e748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.508849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a61f81d-4c6b-493f-8245-8161bd7e115c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.531154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a0fb17bc-9d02-48b3-906a-36ed15d8e748 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:02.532365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a0fb17bc-9d02-48b3-906a-36ed15d8e748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:02.561601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:02.574521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.575176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.575783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.576073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.576708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.578689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.595585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 906/1830] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:02.604525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb17bc-9d02-48b3-906a-36ed15d8e748 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.606326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a0fb17bc-9d02-48b3-906a-36ed15d8e748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:02.606776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 925/1831] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:02 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:02.610760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.612451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:02.627162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36dd33-7a11-406c-916a-33cb31ecde69 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.632769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c32b7-b297-48bf-ac2c-8ae36deb7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.633457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c32b7-b297-48bf-ac2c-8ae36deb7055 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.633726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c32b7-b297-48bf-ac2c-8ae36deb7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.634001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c32b7-b297-48bf-ac2c-8ae36deb7055 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.634778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c32b7-b297-48bf-ac2c-8ae36deb7055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.636405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c32b7-b297-48bf-ac2c-8ae36deb7055 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.647039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 926/1832] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:02.650918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c32b7-b297-48bf-ac2c-8ae36deb7055 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.660614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c32b7-b297-48bf-ac2c-8ae36deb7055 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.665966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf027b-f8fc-427d-be8b-9ab0af5080ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.666558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf027b-f8fc-427d-be8b-9ab0af5080ce admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.666828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf027b-f8fc-427d-be8b-9ab0af5080ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.667099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf027b-f8fc-427d-be8b-9ab0af5080ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.667730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf027b-f8fc-427d-be8b-9ab0af5080ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.667828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb17bc-9d02-48b3-906a-36ed15d8e748 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.669333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf027b-f8fc-427d-be8b-9ab0af5080ce admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.671558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d19dbe-0617-4820-a9e0-419c3f1cf277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.671928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19dbe-0617-4820-a9e0-419c3f1cf277 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.672203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19dbe-0617-4820-a9e0-419c3f1cf277 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.672470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d19dbe-0617-4820-a9e0-419c3f1cf277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.680160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 927/1833] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:02.681709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d19dbe-0617-4820-a9e0-419c3f1cf277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.683311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d19dbe-0617-4820-a9e0-419c3f1cf277 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.684235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf027b-f8fc-427d-be8b-9ab0af5080ce admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.693760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 907/1834] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:02.694269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf027b-f8fc-427d-be8b-9ab0af5080ce admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.698982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79626f1f-86a6-4b1b-8e7e-0b27714fcfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.699621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79626f1f-86a6-4b1b-8e7e-0b27714fcfa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.699898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79626f1f-86a6-4b1b-8e7e-0b27714fcfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.700170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79626f1f-86a6-4b1b-8e7e-0b27714fcfa1 admin admin] PATH_INFO: `/v3/projects/7be5a95fbea6423984cc1d710d99e1d2/users/e54c7de30bc04c73af2f10d5c31b9eb5/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.714618 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79626f1f-86a6-4b1b-8e7e-0b27714fcfa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be5a95fbea6423984cc1d710d99e1d2, user_id=e54c7de30bc04c73af2f10d5c31b9eb5, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.717291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79626f1f-86a6-4b1b-8e7e-0b27714fcfa1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.737171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 928/1835] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:02 2022] PUT /identity/v3/projects/7be5a95fbea6423984cc1d710d99e1d2/users/e54c7de30bc04c73af2f10d5c31b9eb5/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:02.750001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e8e9-70ac-4fdc-be36-ff22b341a125 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.750651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e8e9-70ac-4fdc-be36-ff22b341a125 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.751171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e8e9-70ac-4fdc-be36-ff22b341a125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.785305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-dee4e8e9-70ac-4fdc-be36-ff22b341a125 None None] MFA Rules not processed for user `e54c7de30bc04c73af2f10d5c31b9eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:02.787138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-dee4e8e9-70ac-4fdc-be36-ff22b341a125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:02.843020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:02.857018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 908/1836] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:02.861874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79626f1f-86a6-4b1b-8e7e-0b27714fcfa1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.928382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79626f1f-86a6-4b1b-8e7e-0b27714fcfa1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:02.933439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:02.934297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:02.934669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:02.935032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:02.935768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:02.937799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:02.969600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 929/1837] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:02 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:02.973772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4e8e9-70ac-4fdc-be36-ff22b341a125 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.974993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:02.976683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:02.988505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4e8e9-70ac-4fdc-be36-ff22b341a125 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.048605 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a617-4790-4352-86df-5dd3d3658075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.049244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a617-4790-4352-86df-5dd3d3658075 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.049517 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a617-4790-4352-86df-5dd3d3658075 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.049783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3a617-4790-4352-86df-5dd3d3658075 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.050421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3a617-4790-4352-86df-5dd3d3658075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.052339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3a617-4790-4352-86df-5dd3d3658075 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.053767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b26b1-d336-4a84-b384-2a35c1462996 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.058620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.059273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.059502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.059852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] PATH_INFO: `/v3/projects/f1862262c5964f08899d9863f683fe79` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.061085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1862262c5964f08899d9863f683fe79)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.063016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.074207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 909/1838] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:02 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:03.078078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a617-4790-4352-86df-5dd3d3658075 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:03.089240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3a617-4790-4352-86df-5dd3d3658075 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.091168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:03.096608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620e1af-2135-4208-a436-bfb2943074b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.097318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620e1af-2135-4208-a436-bfb2943074b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.097655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620e1af-2135-4208-a436-bfb2943074b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.097994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620e1af-2135-4208-a436-bfb2943074b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.098875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620e1af-2135-4208-a436-bfb2943074b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.100581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620e1af-2135-4208-a436-bfb2943074b5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.125313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1862262c5964f08899d9863f683fe79', 'request_id': 'req-78646c04-dc72-46b0-8f56-f2f73522bcf7'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.129388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 910/1839] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:03.134264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1862262c5964f08899d9863f683fe79', 'request_id': 'req-78646c04-dc72-46b0-8f56-f2f73522bcf7'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.134406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620e1af-2135-4208-a436-bfb2943074b5 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:03.141841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1862262c5964f08899d9863f683fe79', 'request_id': 'req-78646c04-dc72-46b0-8f56-f2f73522bcf7'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.147045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620e1af-2135-4208-a436-bfb2943074b5 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.160687 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] The token cache is being invalidate because project f1862262c5964f08899d9863f683fe79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:03.161004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.161716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.163300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 930/1840] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:02 2022] DELETE /identity/v3/projects/f1862262c5964f08899d9863f683fe79 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:03.170033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:03.171428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:03.210950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.211535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.212136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.212136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] PATH_INFO: `/v3/projects/35716079417c40639c623c0ef4e1b6fd/users/2eca7d6f32a14b34bca11a454da97777/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.226587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35716079417c40639c623c0ef4e1b6fd, user_id=2eca7d6f32a14b34bca11a454da97777, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.229232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.237698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78646c04-dc72-46b0-8f56-f2f73522bcf7 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.242465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 911/1841] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:03 2022] PUT /identity/v3/projects/35716079417c40639c623c0ef4e1b6fd/users/2eca7d6f32a14b34bca11a454da97777/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:03.246360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:03.248242 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:03.298955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.299597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.299875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.300150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] PATH_INFO: `/v3/users/3c50cb03f508462ba136bbf76ce895b6` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.301320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c50cb03f508462ba136bbf76ce895b6)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.303269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.307902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:03.314285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5580a-cb42-4211-9614-7fc86a543fc2 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.319525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7525d6-fff7-4e58-959a-c83dab34b38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.320238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7525d6-fff7-4e58-959a-c83dab34b38d admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.320465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7525d6-fff7-4e58-959a-c83dab34b38d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.320745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7525d6-fff7-4e58-959a-c83dab34b38d admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.321430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7525d6-fff7-4e58-959a-c83dab34b38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.323650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7525d6-fff7-4e58-959a-c83dab34b38d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.333724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 912/1842] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:03.338146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7525d6-fff7-4e58-959a-c83dab34b38d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:03.348779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7525d6-fff7-4e58-959a-c83dab34b38d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.353439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca6db3-73b0-45c5-8a40-72457d90874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.354090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6db3-73b0-45c5-8a40-72457d90874a admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.354474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6db3-73b0-45c5-8a40-72457d90874a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.354753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6db3-73b0-45c5-8a40-72457d90874a admin admin] PATH_INFO: `/v3/projects/35716079417c40639c623c0ef4e1b6fd/users/2eca7d6f32a14b34bca11a454da97777/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.356635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca6db3-73b0-45c5-8a40-72457d90874a admin admin] RBAC: Authorizing `identity:create_grant(project_id=35716079417c40639c623c0ef4e1b6fd, user_id=2eca7d6f32a14b34bca11a454da97777, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.359121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c50cb03f508462ba136bbf76ce895b6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.359321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca6db3-73b0-45c5-8a40-72457d90874a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.359833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c50cb03f508462ba136bbf76ce895b6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.369453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 913/1843] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:03 2022] PUT /identity/v3/projects/35716079417c40639c623c0ef4e1b6fd/users/2eca7d6f32a14b34bca11a454da97777/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:03.370200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:03.372368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c50cb03f508462ba136bbf76ce895b6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.382429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003454b-a15f-452c-8aa4-dd355414a25c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.382807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003454b-a15f-452c-8aa4-dd355414a25c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.383071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003454b-a15f-452c-8aa4-dd355414a25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.385534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c50cb03f508462ba136bbf76ce895b6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.386298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c50cb03f508462ba136bbf76ce895b6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.399737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 931/1844] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:03 2022] DELETE /identity/v3/users/3c50cb03f508462ba136bbf76ce895b6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:03.411376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e003454b-a15f-452c-8aa4-dd355414a25c None None] MFA Rules not processed for user `2eca7d6f32a14b34bca11a454da97777`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:03.412570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e003454b-a15f-452c-8aa4-dd355414a25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:03.487907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 914/1845] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:03.501152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:03.502986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:03.610264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf636c-4ab0-4f3b-8b26-9c047a262c70 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.620043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf3d105-8a98-4c22-9ff6-d141fd76b0b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.620751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3d105-8a98-4c22-9ff6-d141fd76b0b2 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.621218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3d105-8a98-4c22-9ff6-d141fd76b0b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.621461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3d105-8a98-4c22-9ff6-d141fd76b0b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.623831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-fdf3d105-8a98-4c22-9ff6-d141fd76b0b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:03.696979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3d105-8a98-4c22-9ff6-d141fd76b0b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.698459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf3d105-8a98-4c22-9ff6-d141fd76b0b2 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.708942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 932/1846] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:03.714629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003454b-a15f-452c-8aa4-dd355414a25c None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:03.716301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e003454b-a15f-452c-8aa4-dd355414a25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:03.787869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003454b-a15f-452c-8aa4-dd355414a25c None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:03.792705 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:03.793396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.793669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.793946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] PATH_INFO: `/v3/projects/6880dee6035948febfb316e7320e7b81` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.795110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6880dee6035948febfb316e7320e7b81)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:03.797066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:03.844564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6880dee6035948febfb316e7320e7b81', 'request_id': 'req-10db18b9-1be5-419d-b640-d3001e15b54a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.850041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6880dee6035948febfb316e7320e7b81', 'request_id': 'req-10db18b9-1be5-419d-b640-d3001e15b54a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.856195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6880dee6035948febfb316e7320e7b81', 'request_id': 'req-10db18b9-1be5-419d-b640-d3001e15b54a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.858798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:03.873156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] The token cache is being invalidate because project 6880dee6035948febfb316e7320e7b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:03.873448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.874078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-10db18b9-1be5-419d-b640-d3001e15b54a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:03.875569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 915/1847] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:03 2022] DELETE /identity/v3/projects/6880dee6035948febfb316e7320e7b81 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:03.881126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a967-02d0-4d1e-b4b2-25ce34ee5dc3 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:03.881482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a967-02d0-4d1e-b4b2-25ce34ee5dc3 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:03.881840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a967-02d0-4d1e-b4b2-25ce34ee5dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:03.910722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-3689a967-02d0-4d1e-b4b2-25ce34ee5dc3 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:03.911704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3689a967-02d0-4d1e-b4b2-25ce34ee5dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:03.990871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 933/1848] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:03.999563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c739b6-d7c1-4c17-9a0c-846685d4809a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.000023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c739b6-d7c1-4c17-9a0c-846685d4809a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.000226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c739b6-d7c1-4c17-9a0c-846685d4809a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.048797 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-35c739b6-d7c1-4c17-9a0c-846685d4809a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:04.050339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-35c739b6-d7c1-4c17-9a0c-846685d4809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:04.061124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 916/1849] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:04.064117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689a967-02d0-4d1e-b4b2-25ce34ee5dc3 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.075305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689a967-02d0-4d1e-b4b2-25ce34ee5dc3 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.080553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.081406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.081761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.082242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.082979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:04.083284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.084880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.098086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 934/1850] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:04.102911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c739b6-d7c1-4c17-9a0c-846685d4809a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.111946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c739b6-d7c1-4c17-9a0c-846685d4809a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.116682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ab2ca-53d2-4789-89a5-eab5133440f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.117323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ab2ca-53d2-4789-89a5-eab5133440f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.117598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ab2ca-53d2-4789-89a5-eab5133440f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.117863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ab2ca-53d2-4789-89a5-eab5133440f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.118486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ab2ca-53d2-4789-89a5-eab5133440f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.120367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ab2ca-53d2-4789-89a5-eab5133440f7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.148934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 917/1851] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:04 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:04.153116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.163740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49305646-a3ea-45f5-9dcf-6a1d76aef95f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.180858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:04.224159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf67d0-f4f8-4230-b5d3-48c2ec6a2aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.224693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf67d0-f4f8-4230-b5d3-48c2ec6a2aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.224969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf67d0-f4f8-4230-b5d3-48c2ec6a2aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.225344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf67d0-f4f8-4230-b5d3-48c2ec6a2aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.225976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf67d0-f4f8-4230-b5d3-48c2ec6a2aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.227867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf67d0-f4f8-4230-b5d3-48c2ec6a2aa6 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.252010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 935/1852] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:04 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:04.257884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ab2ca-53d2-4789-89a5-eab5133440f7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.273220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ab2ca-53d2-4789-89a5-eab5133440f7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.277803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa54d14-5007-4ce4-a122-b81544dd5bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.278765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa54d14-5007-4ce4-a122-b81544dd5bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.278765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa54d14-5007-4ce4-a122-b81544dd5bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.279044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa54d14-5007-4ce4-a122-b81544dd5bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.279600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa54d14-5007-4ce4-a122-b81544dd5bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.281302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa54d14-5007-4ce4-a122-b81544dd5bb4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.292371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 918/1853] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:04.296614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf67d0-f4f8-4230-b5d3-48c2ec6a2aa6 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.308586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf67d0-f4f8-4230-b5d3-48c2ec6a2aa6 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.313228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716f756-4451-4486-bffd-69a098260c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.313825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716f756-4451-4486-bffd-69a098260c8e admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.314102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716f756-4451-4486-bffd-69a098260c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.314371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716f756-4451-4486-bffd-69a098260c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.315033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716f756-4451-4486-bffd-69a098260c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.316615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716f756-4451-4486-bffd-69a098260c8e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.327646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 936/1854] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:04.331660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa54d14-5007-4ce4-a122-b81544dd5bb4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.341562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa54d14-5007-4ce4-a122-b81544dd5bb4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.346660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a644f-d404-45f5-84f3-5863278bb2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.347440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a644f-d404-45f5-84f3-5863278bb2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.347760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a644f-d404-45f5-84f3-5863278bb2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.348000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a644f-d404-45f5-84f3-5863278bb2b4 admin admin] PATH_INFO: `/v3/projects/4ffec90c2b614b29a4293110c1317b7e/users/74323ae668c948bebd1bef202e0cad39/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.363288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a644f-d404-45f5-84f3-5863278bb2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ffec90c2b614b29a4293110c1317b7e, user_id=74323ae668c948bebd1bef202e0cad39, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.366785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a644f-d404-45f5-84f3-5863278bb2b4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.381085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 919/1855] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:04 2022] PUT /identity/v3/projects/4ffec90c2b614b29a4293110c1317b7e/users/74323ae668c948bebd1bef202e0cad39/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:04.390971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbb79b-f7e4-4489-a5b5-715024ac3c08 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.391258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbb79b-f7e4-4489-a5b5-715024ac3c08 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.391530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbb79b-f7e4-4489-a5b5-715024ac3c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.419013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-95fbb79b-f7e4-4489-a5b5-715024ac3c08 None None] MFA Rules not processed for user `74323ae668c948bebd1bef202e0cad39`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:04.420220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-95fbb79b-f7e4-4489-a5b5-715024ac3c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:04.483482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 937/1856] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:04.487852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a644f-d404-45f5-84f3-5863278bb2b4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.548280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a644f-d404-45f5-84f3-5863278bb2b4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.554587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bbf3f-41df-439a-8fbc-41f0ea535194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.555480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bbf3f-41df-439a-8fbc-41f0ea535194 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.555760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bbf3f-41df-439a-8fbc-41f0ea535194 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.556032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bbf3f-41df-439a-8fbc-41f0ea535194 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.556665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bbf3f-41df-439a-8fbc-41f0ea535194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.558591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bbf3f-41df-439a-8fbc-41f0ea535194 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.585609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 920/1857] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:04 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:04.589944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbb79b-f7e4-4489-a5b5-715024ac3c08 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.601570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbb79b-f7e4-4489-a5b5-715024ac3c08 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.659589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732405a-629c-4c51-9a0e-afa1724e4f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.660199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732405a-629c-4c51-9a0e-afa1724e4f3f admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.660475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732405a-629c-4c51-9a0e-afa1724e4f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.661443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732405a-629c-4c51-9a0e-afa1724e4f3f admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.662042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732405a-629c-4c51-9a0e-afa1724e4f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.663941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732405a-629c-4c51-9a0e-afa1724e4f3f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.684699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 938/1858] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:04 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:04.688779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bbf3f-41df-439a-8fbc-41f0ea535194 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.699307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bbf3f-41df-439a-8fbc-41f0ea535194 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.705381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5ca7c-5c30-472c-b8be-e7c157f41cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.705381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5ca7c-5c30-472c-b8be-e7c157f41cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.705381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5ca7c-5c30-472c-b8be-e7c157f41cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.705381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5ca7c-5c30-472c-b8be-e7c157f41cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.705724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5ca7c-5c30-472c-b8be-e7c157f41cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.707322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5ca7c-5c30-472c-b8be-e7c157f41cd2 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.719121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 921/1859] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:04.723647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732405a-629c-4c51-9a0e-afa1724e4f3f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.733741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732405a-629c-4c51-9a0e-afa1724e4f3f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.738439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35342979-1b4e-4aac-9f0b-87e45e030671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.739621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342979-1b4e-4aac-9f0b-87e45e030671 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.741103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342979-1b4e-4aac-9f0b-87e45e030671 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.741103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35342979-1b4e-4aac-9f0b-87e45e030671 admin admin] PATH_INFO: `/v3/projects/2f6a0b4450b64378919cd81015eff6c3/users/d6a160e63aa64bc8bd4dccb9e48cd3f3/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.765468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35342979-1b4e-4aac-9f0b-87e45e030671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6a0b4450b64378919cd81015eff6c3, user_id=d6a160e63aa64bc8bd4dccb9e48cd3f3, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.768146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35342979-1b4e-4aac-9f0b-87e45e030671 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.771153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:04.783633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 939/1860] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:04 2022] PUT /identity/v3/projects/2f6a0b4450b64378919cd81015eff6c3/users/d6a160e63aa64bc8bd4dccb9e48cd3f3/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:04.788091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5ca7c-5c30-472c-b8be-e7c157f41cd2 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.798240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5ca7c-5c30-472c-b8be-e7c157f41cd2 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.858393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c21c28-6c75-4447-9b07-b44cf8c3cb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.858921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c21c28-6c75-4447-9b07-b44cf8c3cb7f admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.859201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c21c28-6c75-4447-9b07-b44cf8c3cb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.859475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c21c28-6c75-4447-9b07-b44cf8c3cb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.860106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c21c28-6c75-4447-9b07-b44cf8c3cb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.861741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c21c28-6c75-4447-9b07-b44cf8c3cb7f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.870090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:04.882674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 922/1861] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:04.883904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35342979-1b4e-4aac-9f0b-87e45e030671 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:04.894486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35342979-1b4e-4aac-9f0b-87e45e030671 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:04.899791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:04.900454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.900733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.901008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] PATH_INFO: `/v3/projects/2f6a0b4450b64378919cd81015eff6c3/users/d6a160e63aa64bc8bd4dccb9e48cd3f3/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.902968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6a0b4450b64378919cd81015eff6c3, user_id=d6a160e63aa64bc8bd4dccb9e48cd3f3, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:04.905661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:04.916118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 940/1862] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:04 2022] PUT /identity/v3/projects/2f6a0b4450b64378919cd81015eff6c3/users/d6a160e63aa64bc8bd4dccb9e48cd3f3/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:04.927305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd94690-89f5-4d1b-bc21-7be63f27b128 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:04.927588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd94690-89f5-4d1b-bc21-7be63f27b128 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:04.927860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd94690-89f5-4d1b-bc21-7be63f27b128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:04.955248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-fbd94690-89f5-4d1b-bc21-7be63f27b128 None None] MFA Rules not processed for user `d6a160e63aa64bc8bd4dccb9e48cd3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:04.956447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fbd94690-89f5-4d1b-bc21-7be63f27b128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:05.024670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 923/1863] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:05.035580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:05.037304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:05.096616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd311bf1-90a8-4225-b5e2-2eb1108fd387 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:05.103240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:05.103847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:05.104122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:05.104391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:05.113714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:05.115157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:05.125764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 941/1864] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:05.210825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd94690-89f5-4d1b-bc21-7be63f27b128 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:05.220212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd94690-89f5-4d1b-bc21-7be63f27b128 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:05.223726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:05.224319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:05.224596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:05.224867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:05.281212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:05.282574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:05.291318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 924/1865] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:07.175836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:07.176727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:07.177414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:07.178314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:07.277001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42774985-0f03-4328-af8b-382809dd99cb service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:07.279088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67835bb-415f-414f-82f1-33abfb699c8a service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:07.282178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0fcac-6b1a-4a92-a9e7-00e1df03bb8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:07.282752 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0fcac-6b1a-4a92-a9e7-00e1df03bb8b service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:07.283027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0fcac-6b1a-4a92-a9e7-00e1df03bb8b service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:07.283294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0fcac-6b1a-4a92-a9e7-00e1df03bb8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:07.284999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d8e0fcac-6b1a-4a92-a9e7-00e1df03bb8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:07.292820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d07e5f-f98a-4c20-b4ad-f9bd2cdb3246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:07.293536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07e5f-f98a-4c20-b4ad-f9bd2cdb3246 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:07.293875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07e5f-f98a-4c20-b4ad-f9bd2cdb3246 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:07.294210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d07e5f-f98a-4c20-b4ad-f9bd2cdb3246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:07.301612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0fcac-6b1a-4a92-a9e7-00e1df03bb8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:07.303727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0fcac-6b1a-4a92-a9e7-00e1df03bb8b service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:07.304800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d07e5f-f98a-4c20-b4ad-f9bd2cdb3246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:07.306260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d07e5f-f98a-4c20-b4ad-f9bd2cdb3246 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:07.313335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 942/1866] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:07.314284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 925/1867] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:09.409476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e35a4-63a1-49a9-bfb0-464911178dc1 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:09.409991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e35a4-63a1-49a9-bfb0-464911178dc1 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:09.410082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e35a4-63a1-49a9-bfb0-464911178dc1 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:09.411756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 943/1868] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:49:09 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:49:09.447580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823bc82-02a7-48a2-932d-bb8054f59f94 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:09.448245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823bc82-02a7-48a2-932d-bb8054f59f94 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:09.448347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823bc82-02a7-48a2-932d-bb8054f59f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:09.501877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1823bc82-02a7-48a2-932d-bb8054f59f94 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:09.503527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1823bc82-02a7-48a2-932d-bb8054f59f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:09.540042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:09.552387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530dba5-b93d-4ba3-a81e-2e4eb745cfc0 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:09.552673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530dba5-b93d-4ba3-a81e-2e4eb745cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:09.553009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530dba5-b93d-4ba3-a81e-2e4eb745cfc0 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:09.554290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 944/1869] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:49:09.595083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833edbb7-c646-4372-b87a-ae77773a0d9a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:09.595373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833edbb7-c646-4372-b87a-ae77773a0d9a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:09.595654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833edbb7-c646-4372-b87a-ae77773a0d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:09.613796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 926/1870] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:49:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:09.627389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-833edbb7-c646-4372-b87a-ae77773a0d9a None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:09.628465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-833edbb7-c646-4372-b87a-ae77773a0d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:09.639664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823bc82-02a7-48a2-932d-bb8054f59f94 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:09.640524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 945/1871] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:49:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:09.641795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1823bc82-02a7-48a2-932d-bb8054f59f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:09.655598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823bc82-02a7-48a2-932d-bb8054f59f94 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:09.661782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:09.662888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:09.662888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:09.663084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:09.672752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:09.674018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:09.686417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 927/1872] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:09.715399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833edbb7-c646-4372-b87a-ae77773a0d9a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:09.716836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-833edbb7-c646-4372-b87a-ae77773a0d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:09.740148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833edbb7-c646-4372-b87a-ae77773a0d9a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:09.757151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023f02-1b4e-4c73-beb7-51b068b06943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:09.758498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023f02-1b4e-4c73-beb7-51b068b06943 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:09.759422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023f02-1b4e-4c73-beb7-51b068b06943 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:09.760644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7023f02-1b4e-4c73-beb7-51b068b06943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:09.784814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7023f02-1b4e-4c73-beb7-51b068b06943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:09.790390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7023f02-1b4e-4c73-beb7-51b068b06943 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:09.809347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 946/1873] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:18.030258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:18.036919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:18.072642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43509f-9911-4527-b0e1-565705b5055c service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:18.078542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3dba83-23f2-47f7-8f9a-3c7878d090c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:18.080215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dba83-23f2-47f7-8f9a-3c7878d090c6 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:18.080895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dba83-23f2-47f7-8f9a-3c7878d090c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:18.081277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dba83-23f2-47f7-8f9a-3c7878d090c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:18.084256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5e3dba83-23f2-47f7-8f9a-3c7878d090c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:18.170049 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023f02-1b4e-4c73-beb7-51b068b06943 service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:18.175314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3dba83-23f2-47f7-8f9a-3c7878d090c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:18.177653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3dba83-23f2-47f7-8f9a-3c7878d090c6 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:18.189609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 928/1874] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:18.251797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7023f02-1b4e-4c73-beb7-51b068b06943 service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:18.261693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:18.262601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:18.262984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:18.263408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] PATH_INFO: `/v3/users/74323ae668c948bebd1bef202e0cad39` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:18.264653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=74323ae668c948bebd1bef202e0cad39)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:18.266869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:18.275054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:18.340179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74323ae668c948bebd1bef202e0cad39'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:18.340861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74323ae668c948bebd1bef202e0cad39'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:18.359937 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74323ae668c948bebd1bef202e0cad39'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:18.401555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74323ae668c948bebd1bef202e0cad39'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:18.405414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74323ae668c948bebd1bef202e0cad39'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:18.430477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 947/1875] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:18 2022] DELETE /identity/v3/users/74323ae668c948bebd1bef202e0cad39 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:18.434926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3868e-e5eb-4515-92ca-0195c7aca1d0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:18.435215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3868e-e5eb-4515-92ca-0195c7aca1d0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:18.435490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3868e-e5eb-4515-92ca-0195c7aca1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:18.461192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7bb3868e-e5eb-4515-92ca-0195c7aca1d0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:18.462452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7bb3868e-e5eb-4515-92ca-0195c7aca1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:18.596939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 929/1876] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:18.612026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:18.614188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:18.771219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac517ef7-5d6e-4142-948c-ce88c47fabbd admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:18.784483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c1050-4530-45a0-aa22-15fadc22d6be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:18.787204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c1050-4530-45a0-aa22-15fadc22d6be service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:18.787654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c1050-4530-45a0-aa22-15fadc22d6be service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:18.788321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386c1050-4530-45a0-aa22-15fadc22d6be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:18.820958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c1050-4530-45a0-aa22-15fadc22d6be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:18.824268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386c1050-4530-45a0-aa22-15fadc22d6be service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:18.853141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 948/1877] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:19.376144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3868e-e5eb-4515-92ca-0195c7aca1d0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:19.378342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7bb3868e-e5eb-4515-92ca-0195c7aca1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:19.393535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3868e-e5eb-4515-92ca-0195c7aca1d0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:19.401514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:19.402171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:19.402430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:19.402700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] PATH_INFO: `/v3/projects/4ffec90c2b614b29a4293110c1317b7e` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:19.403833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ffec90c2b614b29a4293110c1317b7e)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:19.405804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:19.446794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:19.464080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ffec90c2b614b29a4293110c1317b7e', 'request_id': 'req-97965560-6e28-4572-9b90-8269d49a5e69'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.470586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ffec90c2b614b29a4293110c1317b7e', 'request_id': 'req-97965560-6e28-4572-9b90-8269d49a5e69'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.477720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ffec90c2b614b29a4293110c1317b7e', 'request_id': 'req-97965560-6e28-4572-9b90-8269d49a5e69'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.491503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] The token cache is being invalidate because project 4ffec90c2b614b29a4293110c1317b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:19.491718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.492370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.494010 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 930/1878] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:19 2022] DELETE /identity/v3/projects/4ffec90c2b614b29a4293110c1317b7e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:19.498289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c1050-4530-45a0-aa22-15fadc22d6be service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:19.506190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:19.507596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:19.508350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386c1050-4530-45a0-aa22-15fadc22d6be service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:19.564508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:19.565279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:19.565630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:19.565989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] PATH_INFO: `/v3/users/d6a160e63aa64bc8bd4dccb9e48cd3f3` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:19.567208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6a160e63aa64bc8bd4dccb9e48cd3f3)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:19.569246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:19.573049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965560-6e28-4572-9b90-8269d49a5e69 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:19.577740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:19.578369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:19.578646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:19.578944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] PATH_INFO: `/v3/users/0df3f44d2ce74312810ec8886a37c97a` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:19.580096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df3f44d2ce74312810ec8886a37c97a)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:19.582557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:19.609478 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:19.635850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6a160e63aa64bc8bd4dccb9e48cd3f3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.636989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6a160e63aa64bc8bd4dccb9e48cd3f3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.640330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df3f44d2ce74312810ec8886a37c97a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.641427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df3f44d2ce74312810ec8886a37c97a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.650764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df3f44d2ce74312810ec8886a37c97a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.654017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6a160e63aa64bc8bd4dccb9e48cd3f3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.670572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df3f44d2ce74312810ec8886a37c97a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.671291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6a160e63aa64bc8bd4dccb9e48cd3f3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.672373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6a160e63aa64bc8bd4dccb9e48cd3f3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.680095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-6ee1ed80-496c-4b1c-8cdd-b0e9df01fa90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df3f44d2ce74312810ec8886a37c97a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:19.687308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 931/1879] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:19 2022] DELETE /identity/v3/users/0df3f44d2ce74312810ec8886a37c97a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:19.693462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8bae9-4422-4cd6-b292-faa929400c40 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:19.693734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8bae9-4422-4cd6-b292-faa929400c40 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:19.694007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8bae9-4422-4cd6-b292-faa929400c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:19.700934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 949/1880] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:19 2022] DELETE /identity/v3/users/d6a160e63aa64bc8bd4dccb9e48cd3f3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:19.720688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0ac8bae9-4422-4cd6-b292-faa929400c40 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:19.721953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0ac8bae9-4422-4cd6-b292-faa929400c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:19.789468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:19.822308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 932/1881] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:19.837517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:19.839459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:19.987602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510009d-cecd-46fa-8878-9106b0233627 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:19.997925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37422a8a-66fa-42b4-8a0e-3cded671ea07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:19.998524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422a8a-66fa-42b4-8a0e-3cded671ea07 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:19.998807 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422a8a-66fa-42b4-8a0e-3cded671ea07 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:19.999378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37422a8a-66fa-42b4-8a0e-3cded671ea07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.016900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37422a8a-66fa-42b4-8a0e-3cded671ea07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.018793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37422a8a-66fa-42b4-8a0e-3cded671ea07 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.031989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 950/1882] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:20.148633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8bae9-4422-4cd6-b292-faa929400c40 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.150544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0ac8bae9-4422-4cd6-b292-faa929400c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:20.172597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8bae9-4422-4cd6-b292-faa929400c40 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.178667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.179519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.179831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.180145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] PATH_INFO: `/v3/projects/2f6a0b4450b64378919cd81015eff6c3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.181338 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f6a0b4450b64378919cd81015eff6c3)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.183254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.290748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f6a0b4450b64378919cd81015eff6c3', 'request_id': 'req-70d451ec-2c65-4179-8c9a-3793184269f2'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.307573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f6a0b4450b64378919cd81015eff6c3', 'request_id': 'req-70d451ec-2c65-4179-8c9a-3793184269f2'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.316086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f6a0b4450b64378919cd81015eff6c3', 'request_id': 'req-70d451ec-2c65-4179-8c9a-3793184269f2'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.334581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] The token cache is being invalidate because project 2f6a0b4450b64378919cd81015eff6c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:20.334891 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.336542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-70d451ec-2c65-4179-8c9a-3793184269f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.341131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 933/1883] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:20 2022] DELETE /identity/v3/projects/2f6a0b4450b64378919cd81015eff6c3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:20.347351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b6c4-160e-4914-b86b-4ac3de37bf20 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.347667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b6c4-160e-4914-b86b-4ac3de37bf20 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.348008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b6c4-160e-4914-b86b-4ac3de37bf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.387466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-8c24b6c4-160e-4914-b86b-4ac3de37bf20 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:20.388696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8c24b6c4-160e-4914-b86b-4ac3de37bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:20.457202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 951/1884] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:20.464997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9341da-5cb1-45b4-85e8-5d440690e625 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.465490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9341da-5cb1-45b4-85e8-5d440690e625 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.465490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9341da-5cb1-45b4-85e8-5d440690e625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.471735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24b6c4-160e-4914-b86b-4ac3de37bf20 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.473498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8c24b6c4-160e-4914-b86b-4ac3de37bf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:20.489744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-cb9341da-5cb1-45b4-85e8-5d440690e625 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:20.489981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24b6c4-160e-4914-b86b-4ac3de37bf20 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.491095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cb9341da-5cb1-45b4-85e8-5d440690e625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:20.495072 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.496586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.497054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.497377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] PATH_INFO: `/v3/projects/c5a7dae40be34e94ab73a9358bddd3e2` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.498772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a7dae40be34e94ab73a9358bddd3e2)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.501522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.504882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 934/1885] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:20.506062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9341da-5cb1-45b4-85e8-5d440690e625 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.516353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9341da-5cb1-45b4-85e8-5d440690e625 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.521027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.521662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.521941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.522734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.523384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:20.524127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.525756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.538701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 935/1886] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:20.543179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.549714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:20.553996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e17831-c7b7-4412-bcf7-15d68a784994 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.554560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a7dae40be34e94ab73a9358bddd3e2', 'request_id': 'req-0c4659ef-1a7b-4cba-bb60-7b255297a456'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.562103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a7dae40be34e94ab73a9358bddd3e2', 'request_id': 'req-0c4659ef-1a7b-4cba-bb60-7b255297a456'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.568848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a7dae40be34e94ab73a9358bddd3e2', 'request_id': 'req-0c4659ef-1a7b-4cba-bb60-7b255297a456'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.581756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] The token cache is being invalidate because project c5a7dae40be34e94ab73a9358bddd3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:20.582044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.582677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0c4659ef-1a7b-4cba-bb60-7b255297a456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:20.584196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 952/1887] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:20 2022] DELETE /identity/v3/projects/c5a7dae40be34e94ab73a9358bddd3e2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:20.588854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e89bcb-83c7-4b1e-8713-7066d2e4e2e1 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.589294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e89bcb-83c7-4b1e-8713-7066d2e4e2e1 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.589636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e89bcb-83c7-4b1e-8713-7066d2e4e2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.608861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f8e89bcb-83c7-4b1e-8713-7066d2e4e2e1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:20.610109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f8e89bcb-83c7-4b1e-8713-7066d2e4e2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:20.616887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.617535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.617818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.618089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.619128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.620592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.655177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 936/1888] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:20.659574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.661399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:20.677259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 953/1889] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:20.685399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b702f9-c7bd-467a-bb57-a6490baa7095 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.685689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b702f9-c7bd-467a-bb57-a6490baa7095 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.685968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b702f9-c7bd-467a-bb57-a6490baa7095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.702148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:20.706388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-16b702f9-c7bd-467a-bb57-a6490baa7095 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:20.708145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-16b702f9-c7bd-467a-bb57-a6490baa7095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:20.742047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1012b5-90f5-4191-9330-405be56ad6c5 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.746885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f4bba-3826-4d65-a4c1-a36c8eb7727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.747790 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f4bba-3826-4d65-a4c1-a36c8eb7727e admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.748168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f4bba-3826-4d65-a4c1-a36c8eb7727e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.748507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f4bba-3826-4d65-a4c1-a36c8eb7727e admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.749179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f4bba-3826-4d65-a4c1-a36c8eb7727e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.751087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f4bba-3826-4d65-a4c1-a36c8eb7727e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.773615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 954/1890] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:20.777234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 937/1891] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:20.778496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f4bba-3826-4d65-a4c1-a36c8eb7727e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.778918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b702f9-c7bd-467a-bb57-a6490baa7095 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.789145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f4bba-3826-4d65-a4c1-a36c8eb7727e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.789580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b702f9-c7bd-467a-bb57-a6490baa7095 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.794165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa24f2-f782-44df-ad32-2cf53340650d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.794701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.794839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa24f2-f782-44df-ad32-2cf53340650d admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.795027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa24f2-f782-44df-ad32-2cf53340650d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.795297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa24f2-f782-44df-ad32-2cf53340650d admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.795386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.795572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.795845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.795936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa24f2-f782-44df-ad32-2cf53340650d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.796599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:20.796906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.797597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa24f2-f782-44df-ad32-2cf53340650d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.798564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.810086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 938/1892] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:20.811909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 955/1893] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:49:20.813931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa24f2-f782-44df-ad32-2cf53340650d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.817074 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.825898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa24f2-f782-44df-ad32-2cf53340650d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.828357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b08d23-ae4d-4980-8db9-8e0fffe8e6c0 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.830633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc5b81-52f7-4c4c-988d-2ff7a9ce4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.831232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc5b81-52f7-4c4c-988d-2ff7a9ce4bce admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.831512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc5b81-52f7-4c4c-988d-2ff7a9ce4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.832467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc5b81-52f7-4c4c-988d-2ff7a9ce4bce admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.832977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc5b81-52f7-4c4c-988d-2ff7a9ce4bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.834713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af52d6c-b9d4-4821-8ac8-dda9ceddc71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.834861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc5b81-52f7-4c4c-988d-2ff7a9ce4bce admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.835416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af52d6c-b9d4-4821-8ac8-dda9ceddc71b admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.835813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af52d6c-b9d4-4821-8ac8-dda9ceddc71b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.836202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af52d6c-b9d4-4821-8ac8-dda9ceddc71b admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.836962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af52d6c-b9d4-4821-8ac8-dda9ceddc71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.838877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af52d6c-b9d4-4821-8ac8-dda9ceddc71b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.851242 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 939/1894] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:20.852898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc5b81-52f7-4c4c-988d-2ff7a9ce4bce admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.863404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc5b81-52f7-4c4c-988d-2ff7a9ce4bce admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:20.873884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 956/1895] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:20.874029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afae76-ff19-44dd-ada3-8862267bda31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:20.874622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afae76-ff19-44dd-ada3-8862267bda31 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.874899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afae76-ff19-44dd-ada3-8862267bda31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.875209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afae76-ff19-44dd-ada3-8862267bda31 admin admin] PATH_INFO: `/v3/projects/c8670788ecb54512ac1a3ba4b5959fce/users/4d48d72c92ff406e88ee9dbf247a933b/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.882723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af52d6c-b9d4-4821-8ac8-dda9ceddc71b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:20.900149 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afae76-ff19-44dd-ada3-8862267bda31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8670788ecb54512ac1a3ba4b5959fce, user_id=4d48d72c92ff406e88ee9dbf247a933b, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:20.904829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afae76-ff19-44dd-ada3-8862267bda31 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:20.920213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 940/1896] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:20 2022] PUT /identity/v3/projects/c8670788ecb54512ac1a3ba4b5959fce/users/4d48d72c92ff406e88ee9dbf247a933b/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:20.931111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb5daf-fea0-40cf-a0d0-c4f2fdc0b6c9 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:20.931397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb5daf-fea0-40cf-a0d0-c4f2fdc0b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:20.931672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb5daf-fea0-40cf-a0d0-c4f2fdc0b6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:20.950614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:20.968475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-fdeb5daf-fea0-40cf-a0d0-c4f2fdc0b6c9 None None] MFA Rules not processed for user `4d48d72c92ff406e88ee9dbf247a933b`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:20.969977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fdeb5daf-fea0-40cf-a0d0-c4f2fdc0b6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.000095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af52d6c-b9d4-4821-8ac8-dda9ceddc71b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.047158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 941/1897] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:21.052066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb5daf-fea0-40cf-a0d0-c4f2fdc0b6c9 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.086326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc6d59-18e4-4107-b9c4-7b36bb0df232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.088318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc6d59-18e4-4107-b9c4-7b36bb0df232 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.088437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc6d59-18e4-4107-b9c4-7b36bb0df232 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.088763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc6d59-18e4-4107-b9c4-7b36bb0df232 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.089567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc6d59-18e4-4107-b9c4-7b36bb0df232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.092371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc6d59-18e4-4107-b9c4-7b36bb0df232 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.135546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 957/1898] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:20 2022] POST /identity/v3/users => generated 303 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:21.141817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc6d59-18e4-4107-b9c4-7b36bb0df232 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.145506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb5daf-fea0-40cf-a0d0-c4f2fdc0b6c9 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.150877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa8b5d-768e-4db9-b85f-aea884041dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.151552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa8b5d-768e-4db9-b85f-aea884041dfc admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.152118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa8b5d-768e-4db9-b85f-aea884041dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.152398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa8b5d-768e-4db9-b85f-aea884041dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.153040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa8b5d-768e-4db9-b85f-aea884041dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.154406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc6d59-18e4-4107-b9c4-7b36bb0df232 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.154967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa8b5d-768e-4db9-b85f-aea884041dfc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.159617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85226-cde4-4bd4-a09e-8ed5135be590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.160333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85226-cde4-4bd4-a09e-8ed5135be590 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.160674 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85226-cde4-4bd4-a09e-8ed5135be590 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.161008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85226-cde4-4bd4-a09e-8ed5135be590 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.161747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca85226-cde4-4bd4-a09e-8ed5135be590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.163869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca85226-cde4-4bd4-a09e-8ed5135be590 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.165495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:21.196767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 958/1899] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:21.202694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85226-cde4-4bd4-a09e-8ed5135be590 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.221505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85226-cde4-4bd4-a09e-8ed5135be590 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.227797 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 942/1900] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:21 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:21.234423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa8b5d-768e-4db9-b85f-aea884041dfc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.330995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bedaf-ab78-497d-a6c0-5749f6b24651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.332999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bedaf-ab78-497d-a6c0-5749f6b24651 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.337601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bedaf-ab78-497d-a6c0-5749f6b24651 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.337601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bedaf-ab78-497d-a6c0-5749f6b24651 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.337601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bedaf-ab78-497d-a6c0-5749f6b24651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.338750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705bedaf-ab78-497d-a6c0-5749f6b24651 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.359069 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 959/1901] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:21.360524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b5ce-8bae-415c-8929-cf6a47d2bcbb None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.361198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b5ce-8bae-415c-8929-cf6a47d2bcbb None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.361621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b5ce-8bae-415c-8929-cf6a47d2bcbb None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.363557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 960/1902] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:49:21.364659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90b5ce-8bae-415c-8929-cf6a47d2bcbb None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.399836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa8b5d-768e-4db9-b85f-aea884041dfc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.405665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90b5ce-8bae-415c-8929-cf6a47d2bcbb None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.416838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b2380-76e2-4dc6-9d43-2c6646a6e9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.417552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2380-76e2-4dc6-9d43-2c6646a6e9bd admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.417835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2380-76e2-4dc6-9d43-2c6646a6e9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.418104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b2380-76e2-4dc6-9d43-2c6646a6e9bd admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.418792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b2380-76e2-4dc6-9d43-2c6646a6e9bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.420693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b2380-76e2-4dc6-9d43-2c6646a6e9bd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.420805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.421620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.422028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.422028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] PATH_INFO: `/v3/projects/ddb7bd4963c74315957e1eb460a19f7d/users/222a47e71a284c60af9ec40feef405b3/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.438916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb7bd4963c74315957e1eb460a19f7d, user_id=222a47e71a284c60af9ec40feef405b3, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.442144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.452693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 943/1903] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:21 2022] POST /identity/v3/users => generated 300 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:21.453963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7d588-3988-4f5e-bd53-fcdfd015a864 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.454348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7d588-3988-4f5e-bd53-fcdfd015a864 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.454718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7d588-3988-4f5e-bd53-fcdfd015a864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.461423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 961/1904] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:21 2022] PUT /identity/v3/projects/ddb7bd4963c74315957e1eb460a19f7d/users/222a47e71a284c60af9ec40feef405b3/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:21.466002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.467585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.478683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-5fc7d588-3988-4f5e-bd53-fcdfd015a864 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:21.480012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5fc7d588-3988-4f5e-bd53-fcdfd015a864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.517166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:21.543352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a341a-c055-44e4-abe6-0ae9f74871cb admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.548720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.549490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.549901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.550380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] PATH_INFO: `/v3/users/50e1bbe5653844efa2a240f21875b667` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.552260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50e1bbe5653844efa2a240f21875b667)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.554352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.555989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 944/1905] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:49:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:21.557020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7d588-3988-4f5e-bd53-fcdfd015a864 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.569154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7d588-3988-4f5e-bd53-fcdfd015a864 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.576274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655ed89-41b1-4818-a66e-7baa47c940e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.577094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655ed89-41b1-4818-a66e-7baa47c940e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.577470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655ed89-41b1-4818-a66e-7baa47c940e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.577829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655ed89-41b1-4818-a66e-7baa47c940e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.578514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655ed89-41b1-4818-a66e-7baa47c940e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.580160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655ed89-41b1-4818-a66e-7baa47c940e3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.592190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 945/1906] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:21.593556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd0bc-515d-4970-916e-07043987db0d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.593895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd0bc-515d-4970-916e-07043987db0d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.594165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd0bc-515d-4970-916e-07043987db0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.619656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d14cd0bc-515d-4970-916e-07043987db0d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:21.621026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d14cd0bc-515d-4970-916e-07043987db0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.626672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50e1bbe5653844efa2a240f21875b667'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:21.627376 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50e1bbe5653844efa2a240f21875b667'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:21.629863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 946/1907] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:21.634483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cd0bc-515d-4970-916e-07043987db0d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.635708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d14cd0bc-515d-4970-916e-07043987db0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.642163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50e1bbe5653844efa2a240f21875b667'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:21.653861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50e1bbe5653844efa2a240f21875b667'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:21.654984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50e1bbe5653844efa2a240f21875b667'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:21.660387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cd0bc-515d-4970-916e-07043987db0d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.667067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.667668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.667949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.668145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 962/1908] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:21 2022] DELETE /identity/v3/users/50e1bbe5653844efa2a240f21875b667 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:21.668266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.668912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.670226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.672305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.684499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.686271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.697537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 947/1909] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:21.698344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.699758 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.746642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248c6c0-f9a9-4e48-98cd-2c70c1deeb83 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.751405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:21.752543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606f3f2-cb71-445a-980f-6a807efd7d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.753438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f3f2-cb71-445a-980f-6a807efd7d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.754380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f3f2-cb71-445a-980f-6a807efd7d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.754675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606f3f2-cb71-445a-980f-6a807efd7d45 admin admin] PATH_INFO: `/v3/projects/2ebc8358fcf447978a1dfc27b53c3097/users/8edaa3fd0eb94960b97cf370e07e4a05/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.767076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf286c-8c78-439f-aa66-09b2b1ac238c service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.769595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f3f2-cb71-445a-980f-6a807efd7d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ebc8358fcf447978a1dfc27b53c3097, user_id=8edaa3fd0eb94960b97cf370e07e4a05, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.772159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606f3f2-cb71-445a-980f-6a807efd7d45 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.785756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 963/1910] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:21 2022] PUT /identity/v3/projects/2ebc8358fcf447978a1dfc27b53c3097/users/8edaa3fd0eb94960b97cf370e07e4a05/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:21.790117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b1980-aa66-49c8-a280-d59264dd363b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.790406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b1980-aa66-49c8-a280-d59264dd363b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.790786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b1980-aa66-49c8-a280-d59264dd363b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.811213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-548b1980-aa66-49c8-a280-d59264dd363b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:21.812732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-548b1980-aa66-49c8-a280-d59264dd363b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.818457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.819069 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.819337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.819606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.821314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.832742 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.834189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.844050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 948/1911] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:21.844859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.848976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:21.854330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb0101-302a-4535-825f-978a1cd8e5a7 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.884233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 964/1912] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:21.895414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b1980-aa66-49c8-a280-d59264dd363b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.896893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-548b1980-aa66-49c8-a280-d59264dd363b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:21.933345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8db1d-0442-4da3-bcc3-c76416062665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.934091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8db1d-0442-4da3-bcc3-c76416062665 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.934442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8db1d-0442-4da3-bcc3-c76416062665 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.934767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8db1d-0442-4da3-bcc3-c76416062665 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.935478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8db1d-0442-4da3-bcc3-c76416062665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.937502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8db1d-0442-4da3-bcc3-c76416062665 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.949043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 949/1913] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:21.956358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8db1d-0442-4da3-bcc3-c76416062665 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:21.968708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8db1d-0442-4da3-bcc3-c76416062665 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.974444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df05c3-2953-4b7c-917f-6ccd84b4a66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.975115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df05c3-2953-4b7c-917f-6ccd84b4a66f admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.975390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df05c3-2953-4b7c-917f-6ccd84b4a66f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.975658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df05c3-2953-4b7c-917f-6ccd84b4a66f admin admin] PATH_INFO: `/v3/projects/2ebc8358fcf447978a1dfc27b53c3097/users/8edaa3fd0eb94960b97cf370e07e4a05/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.978246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df05c3-2953-4b7c-917f-6ccd84b4a66f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ebc8358fcf447978a1dfc27b53c3097, user_id=8edaa3fd0eb94960b97cf370e07e4a05, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.980033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b1980-aa66-49c8-a280-d59264dd363b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:21.981030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df05c3-2953-4b7c-917f-6ccd84b4a66f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:21.983711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:21.984327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:21.984760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:21.985134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:21.993799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 950/1914] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:21 2022] PUT /identity/v3/projects/2ebc8358fcf447978a1dfc27b53c3097/users/8edaa3fd0eb94960b97cf370e07e4a05/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:21.998424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:21.999868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:22.005377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deee7e-cca8-4188-94f7-c230880edd09 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:22.005741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deee7e-cca8-4188-94f7-c230880edd09 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:22.006087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deee7e-cca8-4188-94f7-c230880edd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:22.012805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 965/1915] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:22.046131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-89deee7e-cca8-4188-94f7-c230880edd09 None None] MFA Rules not processed for user `8edaa3fd0eb94960b97cf370e07e4a05`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:22.047481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-89deee7e-cca8-4188-94f7-c230880edd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:22.145164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 951/1916] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:22.156294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:22.158110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:22.298893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd9434-4242-44e5-a6c4-116a2ed3ff45 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:22.394461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da0ca8-abaf-48f6-96b8-3d8d09f02284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:22.394853 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0ca8-abaf-48f6-96b8-3d8d09f02284 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:22.395192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0ca8-abaf-48f6-96b8-3d8d09f02284 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:22.395551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0ca8-abaf-48f6-96b8-3d8d09f02284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:22.407029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da0ca8-abaf-48f6-96b8-3d8d09f02284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:22.408906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da0ca8-abaf-48f6-96b8-3d8d09f02284 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:22.429851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 966/1917] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:22.616799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89deee7e-cca8-4188-94f7-c230880edd09 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:22.618885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-89deee7e-cca8-4188-94f7-c230880edd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:22.641415 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:22.758805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b810-636c-4a93-bcfd-4d85d737d5f9 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:22.758805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b810-636c-4a93-bcfd-4d85d737d5f9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:22.758805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b810-636c-4a93-bcfd-4d85d737d5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:22.791187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89deee7e-cca8-4188-94f7-c230880edd09 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:22.801985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:22.802709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:22.803092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:22.803582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] PATH_INFO: `/v3/projects/288a657ecae14e299870e6d2693d1a16` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:22.805105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=288a657ecae14e299870e6d2693d1a16)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:22.809420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-30f6b810-636c-4a93-bcfd-4d85d737d5f9 None None] MFA Rules not processed for user `222a47e71a284c60af9ec40feef405b3`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:22.811973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-30f6b810-636c-4a93-bcfd-4d85d737d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:22.813840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:22.861583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:22.952563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 967/1918] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:22 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:22.957875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6b810-636c-4a93-bcfd-4d85d737d5f9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:22.959786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '288a657ecae14e299870e6d2693d1a16', 'request_id': 'req-f374adfc-219f-420f-ae02-e9c273823dd9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:22.961374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-30f6b810-636c-4a93-bcfd-4d85d737d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:22.970376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '288a657ecae14e299870e6d2693d1a16', 'request_id': 'req-f374adfc-219f-420f-ae02-e9c273823dd9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:22.984505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6b810-636c-4a93-bcfd-4d85d737d5f9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:22.985734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '288a657ecae14e299870e6d2693d1a16', 'request_id': 'req-f374adfc-219f-420f-ae02-e9c273823dd9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.044120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] The token cache is being invalidate because project 288a657ecae14e299870e6d2693d1a16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:23.044383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.044961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.046954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 952/1919] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:22 2022] DELETE /identity/v3/projects/288a657ecae14e299870e6d2693d1a16 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:23.052859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.056329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:23.083939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.084782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.085354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.085939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.087016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.089997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.137105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 968/1920] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:22 2022] POST /identity/v3/projects => generated 405 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:23.137293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374adfc-219f-420f-ae02-e9c273823dd9 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:23.138847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.140723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:23.235871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f8e41-5d9c-4ac7-91f7-dac046239c9b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:23.241549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fdb74-c028-4355-b8ca-e5b1e303394a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.242267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fdb74-c028-4355-b8ca-e5b1e303394a admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.242613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fdb74-c028-4355-b8ca-e5b1e303394a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.242961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fdb74-c028-4355-b8ca-e5b1e303394a admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.249149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fdb74-c028-4355-b8ca-e5b1e303394a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.249149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fdb74-c028-4355-b8ca-e5b1e303394a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.252200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:23.274992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 969/1921] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:23 2022] POST /identity/v3/users => generated 309 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:23.278639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.279294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.279544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fdb74-c028-4355-b8ca-e5b1e303394a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.279672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.280034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] PATH_INFO: `/v3/users/a30ee97d3471468080e9914c6b7740dd` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.281948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30ee97d3471468080e9914c6b7740dd)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.283908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.292787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fdb74-c028-4355-b8ca-e5b1e303394a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:23.298456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0377f8-a6e9-4ef4-915a-1c2d178008bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.299855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0377f8-a6e9-4ef4-915a-1c2d178008bf admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.300988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0377f8-a6e9-4ef4-915a-1c2d178008bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.301586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0377f8-a6e9-4ef4-915a-1c2d178008bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.302473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0377f8-a6e9-4ef4-915a-1c2d178008bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.304614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0377f8-a6e9-4ef4-915a-1c2d178008bf admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.325546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 970/1922] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:23.327044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0377f8-a6e9-4ef4-915a-1c2d178008bf admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.345107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0377f8-a6e9-4ef4-915a-1c2d178008bf admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:23.355496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.356433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.356623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.356977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] PATH_INFO: `/v3/projects/caddab89f72b4b2fa7f23a1222000ab2/users/aeb0cfcb721e4fcbb6e3be61b85b1482/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.359454 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30ee97d3471468080e9914c6b7740dd'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.360135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30ee97d3471468080e9914c6b7740dd'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.369946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30ee97d3471468080e9914c6b7740dd'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.374304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caddab89f72b4b2fa7f23a1222000ab2, user_id=aeb0cfcb721e4fcbb6e3be61b85b1482, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.377039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.392898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30ee97d3471468080e9914c6b7740dd'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.402479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30ee97d3471468080e9914c6b7740dd'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.409925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 953/1924] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:23 2022] DELETE /identity/v3/users/a30ee97d3471468080e9914c6b7740dd => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:23.410784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 971/1924] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:23 2022] PUT /identity/v3/projects/caddab89f72b4b2fa7f23a1222000ab2/users/aeb0cfcb721e4fcbb6e3be61b85b1482/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:23.410938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.411764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.413962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:23.423991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:23.442385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:23.521346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3b01e-da38-4ff1-a4a3-5fa9fb99b6a3 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:23.528426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e60e1e-9f9a-46b6-9812-d793b19fb393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.529620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478b918-6ef3-48f3-a82d-70e6ab69060f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:23.529749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e60e1e-9f9a-46b6-9812-d793b19fb393 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.532364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e60e1e-9f9a-46b6-9812-d793b19fb393 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.532826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e60e1e-9f9a-46b6-9812-d793b19fb393 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.533666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e60e1e-9f9a-46b6-9812-d793b19fb393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.535731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df013db-c727-49e8-86db-18842772456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.536224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e60e1e-9f9a-46b6-9812-d793b19fb393 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.536384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df013db-c727-49e8-86db-18842772456a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.536793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df013db-c727-49e8-86db-18842772456a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.537091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df013db-c727-49e8-86db-18842772456a admin admin] PATH_INFO: `/v3/users/4d48d72c92ff406e88ee9dbf247a933b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.538353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df013db-c727-49e8-86db-18842772456a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d48d72c92ff406e88ee9dbf247a933b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.540295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df013db-c727-49e8-86db-18842772456a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.553369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 972/1925] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:23.557452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e60e1e-9f9a-46b6-9812-d793b19fb393 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.569051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e60e1e-9f9a-46b6-9812-d793b19fb393 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:23.574840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4d78a-a495-4b7c-8369-c2f749181d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.575525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4d78a-a495-4b7c-8369-c2f749181d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.575801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4d78a-a495-4b7c-8369-c2f749181d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.576067 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4d78a-a495-4b7c-8369-c2f749181d21 admin admin] PATH_INFO: `/v3/projects/caddab89f72b4b2fa7f23a1222000ab2/users/aeb0cfcb721e4fcbb6e3be61b85b1482/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.578215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4d78a-a495-4b7c-8369-c2f749181d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caddab89f72b4b2fa7f23a1222000ab2, user_id=aeb0cfcb721e4fcbb6e3be61b85b1482, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.581113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4d78a-a495-4b7c-8369-c2f749181d21 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.598029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 973/1926] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:23 2022] PUT /identity/v3/projects/caddab89f72b4b2fa7f23a1222000ab2/users/aeb0cfcb721e4fcbb6e3be61b85b1482/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:23.605730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3df013db-c727-49e8-86db-18842772456a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d48d72c92ff406e88ee9dbf247a933b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.613809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3df013db-c727-49e8-86db-18842772456a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d48d72c92ff406e88ee9dbf247a933b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.632355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3df013db-c727-49e8-86db-18842772456a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d48d72c92ff406e88ee9dbf247a933b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.655725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:23.663269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3df013db-c727-49e8-86db-18842772456a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d48d72c92ff406e88ee9dbf247a933b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.678624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3df013db-c727-49e8-86db-18842772456a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d48d72c92ff406e88ee9dbf247a933b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:23.688964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 954/1927] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:23 2022] DELETE /identity/v3/users/4d48d72c92ff406e88ee9dbf247a933b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:23.690644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f1fa-e315-4011-99a4-8de8542f3922 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.691660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f1fa-e315-4011-99a4-8de8542f3922 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.692372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702f1fa-e315-4011-99a4-8de8542f3922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.719230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d702f1fa-e315-4011-99a4-8de8542f3922 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:23.720666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d702f1fa-e315-4011-99a4-8de8542f3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:23.739016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df013db-c727-49e8-86db-18842772456a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.742502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3df013db-c727-49e8-86db-18842772456a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:23.852900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df013db-c727-49e8-86db-18842772456a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:23.860158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:23.861156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:23.861667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:23.862153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] PATH_INFO: `/v3/users/e54c7de30bc04c73af2f10d5c31b9eb5` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:23.864460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e54c7de30bc04c73af2f10d5c31b9eb5)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:23.867725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:23.891975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 974/1928] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:23.913519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702f1fa-e315-4011-99a4-8de8542f3922 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:23.924641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d702f1fa-e315-4011-99a4-8de8542f3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:24.026979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e54c7de30bc04c73af2f10d5c31b9eb5'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:24.027792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e54c7de30bc04c73af2f10d5c31b9eb5'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:24.036672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e54c7de30bc04c73af2f10d5c31b9eb5'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:24.042771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702f1fa-e315-4011-99a4-8de8542f3922 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:24.054249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:24.054878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:24.055154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:24.055422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:24.062108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e54c7de30bc04c73af2f10d5c31b9eb5'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:24.076456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b8b1cf72-ea4e-447d-a61c-ea86447ecd87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e54c7de30bc04c73af2f10d5c31b9eb5'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:24.080782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:24.082402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:24.094886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 955/1929] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:23 2022] DELETE /identity/v3/users/e54c7de30bc04c73af2f10d5c31b9eb5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:24.096474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da478a-0af3-40ca-9308-1cc53327637a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:24.096756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da478a-0af3-40ca-9308-1cc53327637a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:24.097028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da478a-0af3-40ca-9308-1cc53327637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:24.111333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 975/1930] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:24.118722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-84da478a-0af3-40ca-9308-1cc53327637a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:24.127940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-84da478a-0af3-40ca-9308-1cc53327637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:24.214500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 956/1931] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:24.249791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:24.251204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:24.267034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad28830-e34b-427b-b0ed-95dfc5be137e service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:24.337236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:24.379855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0e1f8-993a-4295-8f33-cfe741eb2204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:24.381460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0e1f8-993a-4295-8f33-cfe741eb2204 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:24.382030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0e1f8-993a-4295-8f33-cfe741eb2204 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:24.383210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0e1f8-993a-4295-8f33-cfe741eb2204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:24.397795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0e1f8-993a-4295-8f33-cfe741eb2204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:24.400500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0e1f8-993a-4295-8f33-cfe741eb2204 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:24.418949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 976/1932] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:24.866641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da478a-0af3-40ca-9308-1cc53327637a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:24.869222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-84da478a-0af3-40ca-9308-1cc53327637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:24.886203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:24.892502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da478a-0af3-40ca-9308-1cc53327637a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:24.934958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:24.946865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:24.947248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:24.947555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] PATH_INFO: `/v3/projects/44ac7ae9cfb249318eaab14aa6c7973a` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:24.950699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=44ac7ae9cfb249318eaab14aa6c7973a)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:24.952994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:25.029409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44ac7ae9cfb249318eaab14aa6c7973a', 'request_id': 'req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.050153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44ac7ae9cfb249318eaab14aa6c7973a', 'request_id': 'req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.069319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44ac7ae9cfb249318eaab14aa6c7973a', 'request_id': 'req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.093573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] The token cache is being invalidate because project 44ac7ae9cfb249318eaab14aa6c7973a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:25.094154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.095184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.098866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 957/1933] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:24 2022] DELETE /identity/v3/projects/44ac7ae9cfb249318eaab14aa6c7973a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:25.103441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aa890-7200-4437-aa9e-b8d74f26a1a7 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:25.103949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aa890-7200-4437-aa9e-b8d74f26a1a7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:25.103949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8aa890-7200-4437-aa9e-b8d74f26a1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:25.141304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9a8aa890-7200-4437-aa9e-b8d74f26a1a7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:25.143605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9a8aa890-7200-4437-aa9e-b8d74f26a1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:25.257725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:25.259246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:25.287835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 977/1934] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:25.299601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f51c-86e0-421d-9426-3204cc610d5e None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:25.300276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f51c-86e0-421d-9426-3204cc610d5e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:25.301402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f51c-86e0-421d-9426-3204cc610d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:25.343194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-be87f51c-86e0-421d-9426-3204cc610d5e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:25.344534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-be87f51c-86e0-421d-9426-3204cc610d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:25.360064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 978/1935] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:25.361188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f51c-86e0-421d-9426-3204cc610d5e None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:25.396189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f51c-86e0-421d-9426-3204cc610d5e None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:25.408388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:25.412248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:25.414564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:25.414868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:25.415550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:25.415858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:25.417518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:25.445860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a190c77-38b2-4a1e-bbc7-d5fb65ccce0d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:25.471532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 979/1936] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:25.478049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:25.479066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:25.481339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:25.482531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:25.484368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] PATH_INFO: `/v3/projects/c8670788ecb54512ac1a3ba4b5959fce` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:25.488840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8670788ecb54512ac1a3ba4b5959fce)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:25.494627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:25.501383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076f199-681f-4154-a824-752c94d7aa7d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:25.516541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074468c3-bbf4-4115-8624-f3148cfce852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:25.517433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074468c3-bbf4-4115-8624-f3148cfce852 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:25.518412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074468c3-bbf4-4115-8624-f3148cfce852 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:25.518697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074468c3-bbf4-4115-8624-f3148cfce852 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:25.521851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074468c3-bbf4-4115-8624-f3148cfce852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:25.529711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074468c3-bbf4-4115-8624-f3148cfce852 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:25.613927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8670788ecb54512ac1a3ba4b5959fce', 'request_id': 'req-ec570654-34a0-4718-935f-9b350e800115'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.619502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 980/1937] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:25 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:25.620780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074468c3-bbf4-4115-8624-f3148cfce852 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:25.620957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8670788ecb54512ac1a3ba4b5959fce', 'request_id': 'req-ec570654-34a0-4718-935f-9b350e800115'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.629856 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8670788ecb54512ac1a3ba4b5959fce', 'request_id': 'req-ec570654-34a0-4718-935f-9b350e800115'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.658043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] The token cache is being invalidate because project c8670788ecb54512ac1a3ba4b5959fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:25.658468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.659207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.661585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 958/1938] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:25 2022] DELETE /identity/v3/projects/c8670788ecb54512ac1a3ba4b5959fce => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:25.665857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:25.667945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:25.691752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:25.763836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074468c3-bbf4-4115-8624-f3148cfce852 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:25.771881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf2214-4a84-4d0f-8028-ea8eafa2512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:25.772582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2214-4a84-4d0f-8028-ea8eafa2512e admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:25.772896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2214-4a84-4d0f-8028-ea8eafa2512e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:25.773258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2214-4a84-4d0f-8028-ea8eafa2512e admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:25.773946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf2214-4a84-4d0f-8028-ea8eafa2512e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:25.776999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf2214-4a84-4d0f-8028-ea8eafa2512e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:25.792760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec570654-34a0-4718-935f-9b350e800115 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:25.808121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:25.810157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:25.810842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:25.811573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] PATH_INFO: `/v3/users/8edaa3fd0eb94960b97cf370e07e4a05` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:25.816027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 981/1939] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:25 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:25.816027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cffd8-2a61-40bf-acd7-e5e85b43811f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:25.816027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cffd8-2a61-40bf-acd7-e5e85b43811f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:25.816027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cffd8-2a61-40bf-acd7-e5e85b43811f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:25.824798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8edaa3fd0eb94960b97cf370e07e4a05)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:25.828464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:25.854626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-725cffd8-2a61-40bf-acd7-e5e85b43811f None None] MFA Rules not processed for user `aeb0cfcb721e4fcbb6e3be61b85b1482`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:25.856141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-725cffd8-2a61-40bf-acd7-e5e85b43811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:25.881420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:25.933391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8edaa3fd0eb94960b97cf370e07e4a05'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.935703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8edaa3fd0eb94960b97cf370e07e4a05'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:25.969160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 982/1940] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:25.970101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cffd8-2a61-40bf-acd7-e5e85b43811f None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:25.974469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-725cffd8-2a61-40bf-acd7-e5e85b43811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:25.974864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8edaa3fd0eb94960b97cf370e07e4a05'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.003264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8edaa3fd0eb94960b97cf370e07e4a05'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.004974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cffd8-2a61-40bf-acd7-e5e85b43811f None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:26.016925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8edaa3fd0eb94960b97cf370e07e4a05'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.028026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:26.028026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.028026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.028026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] PATH_INFO: `/v3/projects/7be5a95fbea6423984cc1d710d99e1d2` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.029083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 959/1941] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:25 2022] DELETE /identity/v3/users/8edaa3fd0eb94960b97cf370e07e4a05 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:26.030463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:26.032268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:26.032574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be5a95fbea6423984cc1d710d99e1d2)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:26.035640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:26.132401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58d0e4-de8b-4a74-959e-f9174a8a37d9 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:26.158382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be5a95fbea6423984cc1d710d99e1d2', 'request_id': 'req-05d62ed9-596f-487f-9874-87e0f406855c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.173032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be5a95fbea6423984cc1d710d99e1d2', 'request_id': 'req-05d62ed9-596f-487f-9874-87e0f406855c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.181512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be5a95fbea6423984cc1d710d99e1d2', 'request_id': 'req-05d62ed9-596f-487f-9874-87e0f406855c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.205521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] The token cache is being invalidate because project 7be5a95fbea6423984cc1d710d99e1d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:26.205909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.207554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.211060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 983/1942] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:25 2022] DELETE /identity/v3/projects/7be5a95fbea6423984cc1d710d99e1d2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:26.217014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:26.218493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:26.219561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:26.254821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:26.255477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.255757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.256033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.256687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:26.258358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:26.295352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 960/1943] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:26.296525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:26.310009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:26.359674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204e1e9-7703-4c89-8f81-c0a310e9e964 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:26.366499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2708-9916-4bf2-9a69-7a89aa345e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:26.367113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2708-9916-4bf2-9a69-7a89aa345e1d admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.367418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2708-9916-4bf2-9a69-7a89aa345e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.367659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc2708-9916-4bf2-9a69-7a89aa345e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.368490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc2708-9916-4bf2-9a69-7a89aa345e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:26.370196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc2708-9916-4bf2-9a69-7a89aa345e1d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:26.385429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 961/1944] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:26.388531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2708-9916-4bf2-9a69-7a89aa345e1d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:26.400486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc2708-9916-4bf2-9a69-7a89aa345e1d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:26.400826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62ed9-596f-487f-9874-87e0f406855c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:26.408431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7961f-1c5b-4cf4-a6b0-dc468eea5e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:26.409132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7961f-1c5b-4cf4-a6b0-dc468eea5e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.409421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7961f-1c5b-4cf4-a6b0-dc468eea5e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.411041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:26.411206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7961f-1c5b-4cf4-a6b0-dc468eea5e23 admin admin] PATH_INFO: `/v3/projects/acb15e65b9704657a9a22347abeaa4fa/users/302d27c570da42909db11f613eaf2e3e/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.411755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.412101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.412341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] PATH_INFO: `/v3/users/2eca7d6f32a14b34bca11a454da97777` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.413671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eca7d6f32a14b34bca11a454da97777)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:26.417033 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:26.430235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7961f-1c5b-4cf4-a6b0-dc468eea5e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb15e65b9704657a9a22347abeaa4fa, user_id=302d27c570da42909db11f613eaf2e3e, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:26.433792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7961f-1c5b-4cf4-a6b0-dc468eea5e23 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:26.452004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 962/1945] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:26 2022] PUT /identity/v3/projects/acb15e65b9704657a9a22347abeaa4fa/users/302d27c570da42909db11f613eaf2e3e/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:26.464362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7cbd-0d51-4194-ba10-7e34b24f1676 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.464676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7cbd-0d51-4194-ba10-7e34b24f1676 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.464944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7cbd-0d51-4194-ba10-7e34b24f1676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.490191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:26.499653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eca7d6f32a14b34bca11a454da97777'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.500480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eca7d6f32a14b34bca11a454da97777'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.531523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-4d9e7cbd-0d51-4194-ba10-7e34b24f1676 None None] MFA Rules not processed for user `302d27c570da42909db11f613eaf2e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:26.533256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4d9e7cbd-0d51-4194-ba10-7e34b24f1676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:26.540926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eca7d6f32a14b34bca11a454da97777'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.552004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eca7d6f32a14b34bca11a454da97777'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.552840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eca7d6f32a14b34bca11a454da97777'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:26.575051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 984/1946] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:26 2022] DELETE /identity/v3/users/2eca7d6f32a14b34bca11a454da97777 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:26.622138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 963/1947] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:26.626729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:26.628263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:26.734828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791f245-66e8-4d05-8b1d-5d34a0ade99a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:26.740640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d8cc-01c2-495a-9588-17e9d728148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:26.741318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d8cc-01c2-495a-9588-17e9d728148a admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.741648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d8cc-01c2-495a-9588-17e9d728148a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.741867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9d8cc-01c2-495a-9588-17e9d728148a admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.742566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9d8cc-01c2-495a-9588-17e9d728148a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:26.744941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9d8cc-01c2-495a-9588-17e9d728148a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:26.774851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 985/1948] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:26 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:26.779074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e7cbd-0d51-4194-ba10-7e34b24f1676 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:26.857007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e7cbd-0d51-4194-ba10-7e34b24f1676 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:26.897476 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:26.923202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88d747-d71a-433c-8fb1-1dc463930f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:26.924240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d747-d71a-433c-8fb1-1dc463930f46 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.924362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d747-d71a-433c-8fb1-1dc463930f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.924530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88d747-d71a-433c-8fb1-1dc463930f46 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.925464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88d747-d71a-433c-8fb1-1dc463930f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:26.934707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88d747-d71a-433c-8fb1-1dc463930f46 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:26.961982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 964/1949] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:26 2022] POST /identity/v3/users => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:26.965948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d8cc-01c2-495a-9588-17e9d728148a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:26.977703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9d8cc-01c2-495a-9588-17e9d728148a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:26.984889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8735169-f338-41cf-b366-56826d080124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:26.985721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8735169-f338-41cf-b366-56826d080124 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:26.986110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8735169-f338-41cf-b366-56826d080124 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:26.986586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8735169-f338-41cf-b366-56826d080124 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:26.991314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8735169-f338-41cf-b366-56826d080124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:27.009408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8735169-f338-41cf-b366-56826d080124 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:27.032693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 986/1950] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:27.054203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88d747-d71a-433c-8fb1-1dc463930f46 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:27.061307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582378a-28d6-4aa1-a902-e331e76e8625 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.061601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582378a-28d6-4aa1-a902-e331e76e8625 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.061883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582378a-28d6-4aa1-a902-e331e76e8625 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.063611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 987/1951] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:49:27.072916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88d747-d71a-433c-8fb1-1dc463930f46 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:27.078925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aae97-d281-4b53-8231-61bd0ef382c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:27.081555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aae97-d281-4b53-8231-61bd0ef382c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.081841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aae97-d281-4b53-8231-61bd0ef382c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.082124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aae97-d281-4b53-8231-61bd0ef382c2 admin admin] PATH_INFO: `/v3/projects/b38d15340fe6487aa714f3378b617e2a/users/62c9bf0592534c3ea0bf338d3360e80d/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.100497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aae97-d281-4b53-8231-61bd0ef382c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38d15340fe6487aa714f3378b617e2a, user_id=62c9bf0592534c3ea0bf338d3360e80d, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:27.104134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aae97-d281-4b53-8231-61bd0ef382c2 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:27.106305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d8ef2-92b3-4e9d-8667-4bc499473691 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.106585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d8ef2-92b3-4e9d-8667-4bc499473691 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.106856 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d8ef2-92b3-4e9d-8667-4bc499473691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.142872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 965/1952] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:27 2022] PUT /identity/v3/projects/b38d15340fe6487aa714f3378b617e2a/users/62c9bf0592534c3ea0bf338d3360e80d/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:27.143892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aae97-d281-4b53-8231-61bd0ef382c2 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:27.146326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7b1d8ef2-92b3-4e9d-8667-4bc499473691 None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:27.150568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7b1d8ef2-92b3-4e9d-8667-4bc499473691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:27.264700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aae97-d281-4b53-8231-61bd0ef382c2 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:27.282436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3c6a8-5353-434e-9fa4-b7239340a44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:27.283200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3c6a8-5353-434e-9fa4-b7239340a44a admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.283582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3c6a8-5353-434e-9fa4-b7239340a44a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.284115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3c6a8-5353-434e-9fa4-b7239340a44a admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.284926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3c6a8-5353-434e-9fa4-b7239340a44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:27.286657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3c6a8-5353-434e-9fa4-b7239340a44a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:27.292864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 988/1953] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:49:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:27.306090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d8ef2-92b3-4e9d-8667-4bc499473691 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:27.308678 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7b1d8ef2-92b3-4e9d-8667-4bc499473691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:27.317811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 966/1954] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:27.320894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3c6a8-5353-434e-9fa4-b7239340a44a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:27.328394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d8ef2-92b3-4e9d-8667-4bc499473691 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:27.336538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3c6a8-5353-434e-9fa4-b7239340a44a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:27.338727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbe79c-a35d-4c97-b19d-33ceb5fbe827 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:27.339913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbe79c-a35d-4c97-b19d-33ceb5fbe827 service placement] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.340446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbe79c-a35d-4c97-b19d-33ceb5fbe827 service placement] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.341321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbe79c-a35d-4c97-b19d-33ceb5fbe827 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.343929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:27.344649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.344963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.345282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] PATH_INFO: `/v3/projects/b38d15340fe6487aa714f3378b617e2a/users/62c9bf0592534c3ea0bf338d3360e80d/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.348446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38d15340fe6487aa714f3378b617e2a, user_id=62c9bf0592534c3ea0bf338d3360e80d, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:27.353091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:27.356096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbe79c-a35d-4c97-b19d-33ceb5fbe827 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:27.358030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbe79c-a35d-4c97-b19d-33ceb5fbe827 service placement] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:27.373574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 989/1955] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:27.379832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 967/1956] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:27 2022] PUT /identity/v3/projects/b38d15340fe6487aa714f3378b617e2a/users/62c9bf0592534c3ea0bf338d3360e80d/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:27.391372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d71e8-3c67-406f-8d32-d1002c152da1 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.391655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d71e8-3c67-406f-8d32-d1002c152da1 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.391926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d71e8-3c67-406f-8d32-d1002c152da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.430476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:27.435024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a28d71e8-3c67-406f-8d32-d1002c152da1 None None] MFA Rules not processed for user `62c9bf0592534c3ea0bf338d3360e80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:27.436966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a28d71e8-3c67-406f-8d32-d1002c152da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:27.521980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:27.523906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:27.541167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 990/1957] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:27.553729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d71e8-3c67-406f-8d32-d1002c152da1 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:27.555347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a28d71e8-3c67-406f-8d32-d1002c152da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:27.638500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d71e8-3c67-406f-8d32-d1002c152da1 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:27.652454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:27.653377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.655702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.656096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.656942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df99a-1d81-4ca6-91ff-6d1e0385807e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:27.665911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:27.667557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.667847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.668245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] PATH_INFO: `/v3/projects/2ebc8358fcf447978a1dfc27b53c3097` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.671453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ebc8358fcf447978a1dfc27b53c3097)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:27.671906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:27.674401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:27.676102 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:27.689379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 991/1958] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:27.739730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:27.742140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:27.743941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:27.781252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ff559-2909-4a26-83ed-77e2cf93352d service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:27.793437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ebc8358fcf447978a1dfc27b53c3097', 'request_id': 'req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:27.824554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ebc8358fcf447978a1dfc27b53c3097', 'request_id': 'req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:27.835351 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ebc8358fcf447978a1dfc27b53c3097', 'request_id': 'req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:27.862352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] The token cache is being invalidate because project 2ebc8358fcf447978a1dfc27b53c3097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:27.862676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:27.863576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:27.866318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 968/1959] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:27 2022] DELETE /identity/v3/projects/2ebc8358fcf447978a1dfc27b53c3097 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:27.867442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:27.868864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:27.907932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:27.908600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.908889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.909200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] PATH_INFO: `/v3/projects/35716079417c40639c623c0ef4e1b6fd` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.910963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] RBAC: Authorizing `identity:delete_project(project_id=35716079417c40639c623c0ef4e1b6fd)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:27.913753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:27.941359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e10c3a-a5c1-4870-8b59-12d60dc34eeb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:27.946900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0a662-5d75-4fc1-ae07-92a65d8e0c62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:27.947463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0a662-5d75-4fc1-ae07-92a65d8e0c62 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:27.947740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0a662-5d75-4fc1-ae07-92a65d8e0c62 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:27.948029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0a662-5d75-4fc1-ae07-92a65d8e0c62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:27.950034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f6b0a662-5d75-4fc1-ae07-92a65d8e0c62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:27.971565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:27.979309 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35716079417c40639c623c0ef4e1b6fd', 'request_id': 'req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:27.987121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35716079417c40639c623c0ef4e1b6fd', 'request_id': 'req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:27.994030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35716079417c40639c623c0ef4e1b6fd', 'request_id': 'req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:28.011969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] The token cache is being invalidate because project 35716079417c40639c623c0ef4e1b6fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:28.013170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:28.013396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-24a3ebda-60ed-421c-a77f-96b4d9c9e10d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:28.020979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 992/1960] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:27 2022] DELETE /identity/v3/projects/35716079417c40639c623c0ef4e1b6fd => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:28.020979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e3de1-7232-4440-bb54-7c6e8baa3225 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.020979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e3de1-7232-4440-bb54-7c6e8baa3225 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.020979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e3de1-7232-4440-bb54-7c6e8baa3225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.030314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0a662-5d75-4fc1-ae07-92a65d8e0c62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.034229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0a662-5d75-4fc1-ae07-92a65d8e0c62 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.044649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 969/1961] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:28.045988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9585a5-3d94-4c0e-bf8b-3e3be3709d23 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.046168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9585a5-3d94-4c0e-bf8b-3e3be3709d23 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.046445 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9585a5-3d94-4c0e-bf8b-3e3be3709d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.049683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-743e3de1-7232-4440-bb54-7c6e8baa3225 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:28.051802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-743e3de1-7232-4440-bb54-7c6e8baa3225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.095991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-3f9585a5-3d94-4c0e-bf8b-3e3be3709d23 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:28.097812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3f9585a5-3d94-4c0e-bf8b-3e3be3709d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.139552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 993/1962] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:28.148341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6101e-e94a-4a14-bd8d-cf749f2217b9 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.148639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6101e-e94a-4a14-bd8d-cf749f2217b9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.148917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6101e-e94a-4a14-bd8d-cf749f2217b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.180829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 970/1963] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:28.181936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-10d6101e-e94a-4a14-bd8d-cf749f2217b9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:28.183159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-10d6101e-e94a-4a14-bd8d-cf749f2217b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.188990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90f8f2-ad2a-4fcb-be82-441c49abda36 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.189328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90f8f2-ad2a-4fcb-be82-441c49abda36 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.189575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90f8f2-ad2a-4fcb-be82-441c49abda36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.211838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 994/1964] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:28.218435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6101e-e94a-4a14-bd8d-cf749f2217b9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.235587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6101e-e94a-4a14-bd8d-cf749f2217b9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.242127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.242603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.243036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.243416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.244178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:28.244691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.246822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.252672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-af90f8f2-ad2a-4fcb-be82-441c49abda36 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:28.257426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-af90f8f2-ad2a-4fcb-be82-441c49abda36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.271565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 971/1965] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:28.272991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 995/1966] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:28.273892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ddc2-e743-4858-a340-cba3e6b5f69d None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.274424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ddc2-e743-4858-a340-cba3e6b5f69d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.274834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510ddc2-e743-4858-a340-cba3e6b5f69d None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.276399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 972/1967] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:49:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:49:28.277329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510ddc2-e743-4858-a340-cba3e6b5f69d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.277970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.296604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510ddc2-e743-4858-a340-cba3e6b5f69d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.298582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ef5aaf-2d4e-43cf-9ea6-414bf918f917 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.303347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac9e7-b611-4b41-8ca8-0012b5716d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.304192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac9e7-b611-4b41-8ca8-0012b5716d26 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.305081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac9e7-b611-4b41-8ca8-0012b5716d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.305081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccac9e7-b611-4b41-8ca8-0012b5716d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.306061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac9e7-b611-4b41-8ca8-0012b5716d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.308232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccac9e7-b611-4b41-8ca8-0012b5716d26 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.315065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe778cd-1462-4f7a-b165-20a7a9670ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.315686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:28.315844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe778cd-1462-4f7a-b165-20a7a9670ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.316076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe778cd-1462-4f7a-b165-20a7a9670ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.316385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe778cd-1462-4f7a-b165-20a7a9670ee4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.317045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe778cd-1462-4f7a-b165-20a7a9670ee4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:28.317463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe778cd-1462-4f7a-b165-20a7a9670ee4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.319097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe778cd-1462-4f7a-b165-20a7a9670ee4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.347630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 996/1968] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:28.347630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453fe93-c1b3-4701-a21a-f610de582e26 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.347630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453fe93-c1b3-4701-a21a-f610de582e26 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.347630 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453fe93-c1b3-4701-a21a-f610de582e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.363987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 973/1969] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:28.363987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac9e7-b611-4b41-8ca8-0012b5716d26 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.395579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4453fe93-c1b3-4701-a21a-f610de582e26 None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:28.400895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4453fe93-c1b3-4701-a21a-f610de582e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.486096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccac9e7-b611-4b41-8ca8-0012b5716d26 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.511170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.512426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.513109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.513368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.514036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.516048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.576378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 997/1970] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:28.577196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453fe93-c1b3-4701-a21a-f610de582e26 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.601628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453fe93-c1b3-4701-a21a-f610de582e26 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.607284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.608055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.608596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.609091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.610551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.613154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.615236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 974/1971] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/projects => generated 377 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:28.617106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.618516 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.643601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 998/1972] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/users => generated 293 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:28.645672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.647633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.697287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813090e-5f18-46e2-9504-b80e0a8c8c6a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.718112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:28.752365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8010f1c-edfe-46a1-a47d-65a9dec512a7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.757277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.757985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.758267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.758611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] PATH_INFO: `/v3/users/222a47e71a284c60af9ec40feef405b3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.762064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] RBAC: Authorizing `identity:delete_user(user_id=222a47e71a284c60af9ec40feef405b3)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.764638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.778154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fd456-9873-4cb8-a9f1-c1d925ab0ee2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.778857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd456-9873-4cb8-a9f1-c1d925ab0ee2 service placement] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.779214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd456-9873-4cb8-a9f1-c1d925ab0ee2 service placement] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.779558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd456-9873-4cb8-a9f1-c1d925ab0ee2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.790633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fd456-9873-4cb8-a9f1-c1d925ab0ee2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.792455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fd456-9873-4cb8-a9f1-c1d925ab0ee2 service placement] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.804138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 999/1973] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:28.805357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fd456-9873-4cb8-a9f1-c1d925ab0ee2 service placement] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.817912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fd456-9873-4cb8-a9f1-c1d925ab0ee2 service placement] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.823656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0a176-9354-4a21-bbca-1044de5cfaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.824401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0a176-9354-4a21-bbca-1044de5cfaf1 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.824740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0a176-9354-4a21-bbca-1044de5cfaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.825091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0a176-9354-4a21-bbca-1044de5cfaf1 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.825786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0a176-9354-4a21-bbca-1044de5cfaf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.829262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0a176-9354-4a21-bbca-1044de5cfaf1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.829948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222a47e71a284c60af9ec40feef405b3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:28.830669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222a47e71a284c60af9ec40feef405b3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:28.850665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222a47e71a284c60af9ec40feef405b3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:28.863132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1000/1974] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:28 2022] POST /identity/v3/users => generated 289 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:28.863132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0a176-9354-4a21-bbca-1044de5cfaf1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.877230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0a176-9354-4a21-bbca-1044de5cfaf1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.880388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222a47e71a284c60af9ec40feef405b3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:28.882231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:28.889401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.890353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.890353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.890813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.893113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.895052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:28.896676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222a47e71a284c60af9ec40feef405b3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:28.903754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 975/1975] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:28 2022] DELETE /identity/v3/users/222a47e71a284c60af9ec40feef405b3 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:28.904724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.906371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.914908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1001/1976] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:28.917323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:28.919289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:28.932473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5677c-0443-47df-b050-be050c2383e6 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.991201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970df74a-424e-4b8d-a0f1-b12ae5a2757f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:28.996049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80499611-c21a-490e-8c89-9c2b4cc9999f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:28.996669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80499611-c21a-490e-8c89-9c2b4cc9999f admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:28.996951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80499611-c21a-490e-8c89-9c2b4cc9999f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:28.997285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80499611-c21a-490e-8c89-9c2b4cc9999f admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:28.998188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80499611-c21a-490e-8c89-9c2b4cc9999f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:28.999786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80499611-c21a-490e-8c89-9c2b4cc9999f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.007908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dd1eb-95ea-4a08-9fdf-85b7af891a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.008522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd1eb-95ea-4a08-9fdf-85b7af891a31 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.008797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd1eb-95ea-4a08-9fdf-85b7af891a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.009077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd1eb-95ea-4a08-9fdf-85b7af891a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.009719 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dd1eb-95ea-4a08-9fdf-85b7af891a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.010834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 976/1977] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:29.011709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dd1eb-95ea-4a08-9fdf-85b7af891a31 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.014587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80499611-c21a-490e-8c89-9c2b4cc9999f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.024931 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1002/1978] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:29.026840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80499611-c21a-490e-8c89-9c2b4cc9999f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.029193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dd1eb-95ea-4a08-9fdf-85b7af891a31 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.031855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538f95-6433-45c2-abb4-d7518b86d0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.032463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538f95-6433-45c2-abb4-d7518b86d0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.032746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538f95-6433-45c2-abb4-d7518b86d0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.033023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538f95-6433-45c2-abb4-d7518b86d0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.035765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2538f95-6433-45c2-abb4-d7518b86d0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.035765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2538f95-6433-45c2-abb4-d7518b86d0b8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.041618 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dd1eb-95ea-4a08-9fdf-85b7af891a31 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.046457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e3cead-a2a1-4897-a537-ca51c4c2cab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.046655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 977/1979] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:29.047132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3cead-a2a1-4897-a537-ca51c4c2cab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.047404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3cead-a2a1-4897-a537-ca51c4c2cab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.047701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3cead-a2a1-4897-a537-ca51c4c2cab0 admin admin] PATH_INFO: `/v3/projects/15de188cf537408895579b4fe30ebec1/users/a7b7352e0ff74b18a3f9ef9954986973/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.051864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538f95-6433-45c2-abb4-d7518b86d0b8 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.067355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538f95-6433-45c2-abb4-d7518b86d0b8 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.070287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3cead-a2a1-4897-a537-ca51c4c2cab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15de188cf537408895579b4fe30ebec1, user_id=a7b7352e0ff74b18a3f9ef9954986973, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.072490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.073707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.073707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.073998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] PATH_INFO: `/v3/projects/ee6904f697d34b7da45fe23f883d2084/users/2d31629a654247f2916cd09ec3f3681e/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.075125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e3cead-a2a1-4897-a537-ca51c4c2cab0 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.093960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee6904f697d34b7da45fe23f883d2084, user_id=2d31629a654247f2916cd09ec3f3681e, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.097562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.103821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1003/1980] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:29 2022] PUT /identity/v3/projects/15de188cf537408895579b4fe30ebec1/users/a7b7352e0ff74b18a3f9ef9954986973/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:29.115220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a470000-f5ea-406b-87ca-c9608ed5fad3 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.115557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a470000-f5ea-406b-87ca-c9608ed5fad3 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.115794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a470000-f5ea-406b-87ca-c9608ed5fad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.137184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 978/1981] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:29 2022] PUT /identity/v3/projects/ee6904f697d34b7da45fe23f883d2084/users/2d31629a654247f2916cd09ec3f3681e/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:29.138743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.142858 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:29.150452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2a470000-f5ea-406b-87ca-c9608ed5fad3 None None] MFA Rules not processed for user `a7b7352e0ff74b18a3f9ef9954986973`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:29.151980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2a470000-f5ea-406b-87ca-c9608ed5fad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:29.246352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:29.258454 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f6d4a-8afb-4702-a047-746e39eec86c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.268913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.270514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.271072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.271596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] PATH_INFO: `/v3/projects/ddb7bd4963c74315957e1eb460a19f7d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.273158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb7bd4963c74315957e1eb460a19f7d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.277335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.284007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1004/1982] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:29.285443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229d9c-525e-4425-a8ad-2a568135f9e7 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.285714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229d9c-525e-4425-a8ad-2a568135f9e7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.285985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24229d9c-525e-4425-a8ad-2a568135f9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.321306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-24229d9c-525e-4425-a8ad-2a568135f9e7 None None] MFA Rules not processed for user `2d31629a654247f2916cd09ec3f3681e`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:29.322516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-24229d9c-525e-4425-a8ad-2a568135f9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:29.339248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb7bd4963c74315957e1eb460a19f7d', 'request_id': 'req-d7190e15-df54-4faf-a074-c2cbb63d265e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.341566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:29.349806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb7bd4963c74315957e1eb460a19f7d', 'request_id': 'req-d7190e15-df54-4faf-a074-c2cbb63d265e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.360355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb7bd4963c74315957e1eb460a19f7d', 'request_id': 'req-d7190e15-df54-4faf-a074-c2cbb63d265e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.376612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] The token cache is being invalidate because project ddb7bd4963c74315957e1eb460a19f7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:29.376952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.377757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.380408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 979/1983] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:29 2022] DELETE /identity/v3/projects/ddb7bd4963c74315957e1eb460a19f7d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:29.381258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.382593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:29.414694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1005/1984] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:29.416775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24229d9c-525e-4425-a8ad-2a568135f9e7 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.418482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-24229d9c-525e-4425-a8ad-2a568135f9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:29.445441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7190e15-df54-4faf-a074-c2cbb63d265e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.450598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.451219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.451500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.451774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.453820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:29.495102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24229d9c-525e-4425-a8ad-2a568135f9e7 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.500199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.500991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.501357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.501638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] PATH_INFO: `/v3/users/aeb0cfcb721e4fcbb6e3be61b85b1482` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.502775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb0cfcb721e4fcbb6e3be61b85b1482)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.505383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.525082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.526775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.537519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 980/1985] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:29.538292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.539839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:29.556336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbacd0d-b1f7-4b83-8b20-3e9354500e23 service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.561988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.562695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.563466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.563466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.564221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.566810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.568704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb0cfcb721e4fcbb6e3be61b85b1482'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.569642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb0cfcb721e4fcbb6e3be61b85b1482'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.583934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb0cfcb721e4fcbb6e3be61b85b1482'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.598518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb0cfcb721e4fcbb6e3be61b85b1482'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.599411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb0cfcb721e4fcbb6e3be61b85b1482'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:29.601236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:29.627949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 981/1986] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:29 2022] POST /identity/v3/projects => generated 377 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:29.629466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.649389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:29.649732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1006/1987] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:29 2022] DELETE /identity/v3/users/aeb0cfcb721e4fcbb6e3be61b85b1482 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:29.783344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459bd05-2651-40cc-a8f4-faeb7e08ca7f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.823881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:29.879765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908066c9-6279-451a-b5e6-3f5064b2dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.880574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908066c9-6279-451a-b5e6-3f5064b2dc23 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.881192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908066c9-6279-451a-b5e6-3f5064b2dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.881853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908066c9-6279-451a-b5e6-3f5064b2dc23 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.882741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908066c9-6279-451a-b5e6-3f5064b2dc23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.885332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908066c9-6279-451a-b5e6-3f5064b2dc23 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.925131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 982/1988] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:29 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:29.928226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:29.941419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435ed43-19e2-4879-b7a7-f4f607eeb643 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:29.948172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a242908-f369-4c0e-bfa3-e64d4632ac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:29.949209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242908-f369-4c0e-bfa3-e64d4632ac2e admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:29.949720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242908-f369-4c0e-bfa3-e64d4632ac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:29.950445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a242908-f369-4c0e-bfa3-e64d4632ac2e admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:29.958093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a242908-f369-4c0e-bfa3-e64d4632ac2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:29.971334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a242908-f369-4c0e-bfa3-e64d4632ac2e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:29.997973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1007/1989] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:29.999775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908066c9-6279-451a-b5e6-3f5064b2dc23 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.014151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908066c9-6279-451a-b5e6-3f5064b2dc23 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.027073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.027798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.028171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.028555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.029466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.031299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.048593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 983/1990] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:30.053590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a242908-f369-4c0e-bfa3-e64d4632ac2e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.065040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a242908-f369-4c0e-bfa3-e64d4632ac2e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.072780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f48723-81e0-4883-96b3-e9e759e495b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.073779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f48723-81e0-4883-96b3-e9e759e495b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.074156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f48723-81e0-4883-96b3-e9e759e495b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.074522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f48723-81e0-4883-96b3-e9e759e495b1 admin admin] PATH_INFO: `/v3/projects/24d0607c961b47a2925ab6dd57a50143/users/41c05e9355f244d0b421cfdd170988cf/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.105640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f48723-81e0-4883-96b3-e9e759e495b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d0607c961b47a2925ab6dd57a50143, user_id=41c05e9355f244d0b421cfdd170988cf, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.109166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f48723-81e0-4883-96b3-e9e759e495b1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.129450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1008/1991] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:30 2022] PUT /identity/v3/projects/24d0607c961b47a2925ab6dd57a50143/users/41c05e9355f244d0b421cfdd170988cf/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:30.133594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.137435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:30.148236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a3583-ae62-49e7-b428-c3d9f79c5a07 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.148422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a3583-ae62-49e7-b428-c3d9f79c5a07 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.148764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a3583-ae62-49e7-b428-c3d9f79c5a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.165953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea39d-035b-4897-8f4f-2804e713ef66 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.226707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6d8a3583-ae62-49e7-b428-c3d9f79c5a07 None None] MFA Rules not processed for user `41c05e9355f244d0b421cfdd170988cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:30.228506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6d8a3583-ae62-49e7-b428-c3d9f79c5a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:30.302027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.303054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.303341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.303959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] PATH_INFO: `/v3/projects/caddab89f72b4b2fa7f23a1222000ab2` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.305869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=caddab89f72b4b2fa7f23a1222000ab2)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.308609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.337604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1009/1992] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:30.341707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a3583-ae62-49e7-b428-c3d9f79c5a07 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.361941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a3583-ae62-49e7-b428-c3d9f79c5a07 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.402640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caddab89f72b4b2fa7f23a1222000ab2', 'request_id': 'req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:30.419798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caddab89f72b4b2fa7f23a1222000ab2', 'request_id': 'req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:30.445340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caddab89f72b4b2fa7f23a1222000ab2', 'request_id': 'req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:30.462609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] The token cache is being invalidate because project caddab89f72b4b2fa7f23a1222000ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:30.463062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:30.463882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-2d4e349b-e5d6-4f4c-8545-87bbce8e2f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:30.466140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 984/1993] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:30 2022] DELETE /identity/v3/projects/caddab89f72b4b2fa7f23a1222000ab2 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:30.471012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f38f-3bd7-4e9d-be61-43519ac29f8c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.471557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f38f-3bd7-4e9d-be61-43519ac29f8c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.472133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f38f-3bd7-4e9d-be61-43519ac29f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.496624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8e63f38f-3bd7-4e9d-be61-43519ac29f8c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:30.497053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.498199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.498651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.499112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.499424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8e63f38f-3bd7-4e9d-be61-43519ac29f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:30.500183 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.502677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.518462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 985/1994] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:30.521554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda63c99-23b1-447a-a309-bb27a45ccb52 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.522081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda63c99-23b1-447a-a309-bb27a45ccb52 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.522733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda63c99-23b1-447a-a309-bb27a45ccb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.550541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-fda63c99-23b1-447a-a309-bb27a45ccb52 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:30.551783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fda63c99-23b1-447a-a309-bb27a45ccb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:30.560138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1010/1995] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:30 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:30.563946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.566411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:30.575678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 986/1996] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:30.575678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda63c99-23b1-447a-a309-bb27a45ccb52 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.596519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda63c99-23b1-447a-a309-bb27a45ccb52 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.601125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:30.630986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:30.641871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d619f93-a064-4819-a9d8-40c13808c6e0 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.648003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de078cb5-0429-4e0a-9570-1fe05d02c1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.648758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de078cb5-0429-4e0a-9570-1fe05d02c1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.649422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de078cb5-0429-4e0a-9570-1fe05d02c1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.649704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de078cb5-0429-4e0a-9570-1fe05d02c1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.650412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de078cb5-0429-4e0a-9570-1fe05d02c1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.652497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de078cb5-0429-4e0a-9570-1fe05d02c1b6 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.682276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1011/1997] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:30 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:30.687212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de078cb5-0429-4e0a-9570-1fe05d02c1b6 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.692475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.693120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.693407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.693682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.694610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:30.694919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.696829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.699199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de078cb5-0429-4e0a-9570-1fe05d02c1b6 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.707233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c03c5-71be-47b2-8ffc-2f0c23d9be48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.708407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c03c5-71be-47b2-8ffc-2f0c23d9be48 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.708697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c03c5-71be-47b2-8ffc-2f0c23d9be48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.709022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c03c5-71be-47b2-8ffc-2f0c23d9be48 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.710192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c03c5-71be-47b2-8ffc-2f0c23d9be48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.711503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 987/1998] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:30.712016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c03c5-71be-47b2-8ffc-2f0c23d9be48 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.715743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.724471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1012/1999] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:30.727488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72f9f1-f03b-4076-9c47-2b0b9b0f02bb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.729423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c03c5-71be-47b2-8ffc-2f0c23d9be48 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.732955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964df01-e789-471e-ad18-64bd0c254ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.733603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964df01-e789-471e-ad18-64bd0c254ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.733886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964df01-e789-471e-ad18-64bd0c254ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.734153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964df01-e789-471e-ad18-64bd0c254ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.734786 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964df01-e789-471e-ad18-64bd0c254ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.736733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964df01-e789-471e-ad18-64bd0c254ed0 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.740578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c03c5-71be-47b2-8ffc-2f0c23d9be48 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.745898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb123c-30e9-4ea1-8838-ba05a7ac7577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.746622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb123c-30e9-4ea1-8838-ba05a7ac7577 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.747080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb123c-30e9-4ea1-8838-ba05a7ac7577 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.747695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb123c-30e9-4ea1-8838-ba05a7ac7577 admin admin] PATH_INFO: `/v3/projects/c2f98ab4a21c4e44a58ba2b93115535a/users/8ec618778cad4388a038b24451c774c9/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.768203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 988/2000] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:30 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:30.772310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964df01-e789-471e-ad18-64bd0c254ed0 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.773665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb123c-30e9-4ea1-8838-ba05a7ac7577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f98ab4a21c4e44a58ba2b93115535a, user_id=8ec618778cad4388a038b24451c774c9, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.776545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb123c-30e9-4ea1-8838-ba05a7ac7577 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.793187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1013/2001] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:30 2022] PUT /identity/v3/projects/c2f98ab4a21c4e44a58ba2b93115535a/users/8ec618778cad4388a038b24451c774c9/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:30.795514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb123c-30e9-4ea1-8838-ba05a7ac7577 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.852478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964df01-e789-471e-ad18-64bd0c254ed0 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.868934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb123c-30e9-4ea1-8838-ba05a7ac7577 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.874737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ef034-36ee-49dc-ab29-5ee853b920f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.875351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef034-36ee-49dc-ab29-5ee853b920f1 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.875625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef034-36ee-49dc-ab29-5ee853b920f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.876194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef034-36ee-49dc-ab29-5ee853b920f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.876833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ef034-36ee-49dc-ab29-5ee853b920f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.879133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ef034-36ee-49dc-ab29-5ee853b920f1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.892356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1014/2002] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:30.897309 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ef034-36ee-49dc-ab29-5ee853b920f1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.909961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ef034-36ee-49dc-ab29-5ee853b920f1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.915259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb81d55-011d-47e2-9837-a6c7ea375bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.915918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81d55-011d-47e2-9837-a6c7ea375bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.916194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81d55-011d-47e2-9837-a6c7ea375bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.916465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb81d55-011d-47e2-9837-a6c7ea375bfa admin admin] PATH_INFO: `/v3/projects/c2f98ab4a21c4e44a58ba2b93115535a/users/8ec618778cad4388a038b24451c774c9/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.922372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb81d55-011d-47e2-9837-a6c7ea375bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f98ab4a21c4e44a58ba2b93115535a, user_id=8ec618778cad4388a038b24451c774c9, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.923193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f12b32-fb85-44a2-8462-a2d2afdc0687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:30.923805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f12b32-fb85-44a2-8462-a2d2afdc0687 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.924114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f12b32-fb85-44a2-8462-a2d2afdc0687 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.924406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f12b32-fb85-44a2-8462-a2d2afdc0687 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.925590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f12b32-fb85-44a2-8462-a2d2afdc0687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:30.925753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb81d55-011d-47e2-9837-a6c7ea375bfa admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.927621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f12b32-fb85-44a2-8462-a2d2afdc0687 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:30.936378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1015/2003] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:30 2022] PUT /identity/v3/projects/c2f98ab4a21c4e44a58ba2b93115535a/users/8ec618778cad4388a038b24451c774c9/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:30.948289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731caaa-a95b-405e-bbe5-7fdbfd18af4f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:30.948626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731caaa-a95b-405e-bbe5-7fdbfd18af4f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:30.948907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731caaa-a95b-405e-bbe5-7fdbfd18af4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:30.951705 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 989/2004] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:30 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:30.955011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f12b32-fb85-44a2-8462-a2d2afdc0687 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:30.964918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f12b32-fb85-44a2-8462-a2d2afdc0687 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:30.979529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-3731caaa-a95b-405e-bbe5-7fdbfd18af4f None None] MFA Rules not processed for user `8ec618778cad4388a038b24451c774c9`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:30.980804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3731caaa-a95b-405e-bbe5-7fdbfd18af4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:31.014233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:31.057357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:31.067503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c377a8-253e-4035-ac88-ffbc33f537fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.068113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c377a8-253e-4035-ac88-ffbc33f537fb admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.068392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c377a8-253e-4035-ac88-ffbc33f537fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.068661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c377a8-253e-4035-ac88-ffbc33f537fb admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.069327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c377a8-253e-4035-ac88-ffbc33f537fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.070920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c377a8-253e-4035-ac88-ffbc33f537fb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.077518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1016/2005] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:31.080609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 990/2006] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:31.082916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731caaa-a95b-405e-bbe5-7fdbfd18af4f None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.084326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3731caaa-a95b-405e-bbe5-7fdbfd18af4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:31.084441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c377a8-253e-4035-ac88-ffbc33f537fb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.095298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c377a8-253e-4035-ac88-ffbc33f537fb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.097667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731caaa-a95b-405e-bbe5-7fdbfd18af4f None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.100545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.101172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.101450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.101831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.102345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.102440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.102990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.103267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.104162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] PATH_INFO: `/v3/users/302d27c570da42909db11f613eaf2e3e` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.104278 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.104761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] RBAC: Authorizing `identity:delete_user(user_id=302d27c570da42909db11f613eaf2e3e)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.106768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.120574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 991/2007] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:31.121486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.123500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:31.166487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302d27c570da42909db11f613eaf2e3e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:31.167426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302d27c570da42909db11f613eaf2e3e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:31.181390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302d27c570da42909db11f613eaf2e3e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:31.189900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302d27c570da42909db11f613eaf2e3e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:31.191159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302d27c570da42909db11f613eaf2e3e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:31.197425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd758b-1514-4eea-96f7-c86bfd4996dc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.201214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e869f8-2295-40da-a166-123432f1696e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.201820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e869f8-2295-40da-a166-123432f1696e service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.202106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e869f8-2295-40da-a166-123432f1696e service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.202477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e869f8-2295-40da-a166-123432f1696e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.204229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-49e869f8-2295-40da-a166-123432f1696e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:31.206179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1017/2008] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:31 2022] DELETE /identity/v3/users/302d27c570da42909db11f613eaf2e3e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:31.206860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.208217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:31.278306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e869f8-2295-40da-a166-123432f1696e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.279748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e869f8-2295-40da-a166-123432f1696e service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.281680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ef187-8ae4-4eda-a3f3-6c1fd980505b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.287787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e2c24-98c2-4282-bb00-67ad478c365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.288525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e2c24-98c2-4282-bb00-67ad478c365a admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.288870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e2c24-98c2-4282-bb00-67ad478c365a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.289243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e2c24-98c2-4282-bb00-67ad478c365a admin admin] PATH_INFO: `/v3/projects/86274b6395e041e79c291d3ffbe904c8/users/7e6b6288f32048ca8cc518463f4520b5/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.289858 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 992/2009] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:31.300975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd036a-1cab-4c79-a319-544c77056b5e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.301643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd036a-1cab-4c79-a319-544c77056b5e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.302161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd036a-1cab-4c79-a319-544c77056b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.308906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e2c24-98c2-4282-bb00-67ad478c365a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86274b6395e041e79c291d3ffbe904c8, user_id=7e6b6288f32048ca8cc518463f4520b5, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.312997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e2c24-98c2-4282-bb00-67ad478c365a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.342380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b5fd036a-1cab-4c79-a319-544c77056b5e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:31.343883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b5fd036a-1cab-4c79-a319-544c77056b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:31.356869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1018/2010] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:31 2022] PUT /identity/v3/projects/86274b6395e041e79c291d3ffbe904c8/users/7e6b6288f32048ca8cc518463f4520b5/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:31.369377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5f79-cb14-4610-b0d1-546dc39afa96 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.369657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5f79-cb14-4610-b0d1-546dc39afa96 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.369943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f5f79-cb14-4610-b0d1-546dc39afa96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.401433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f40f5f79-cb14-4610-b0d1-546dc39afa96 None None] MFA Rules not processed for user `7e6b6288f32048ca8cc518463f4520b5`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:31.403068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f40f5f79-cb14-4610-b0d1-546dc39afa96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:31.424854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 993/2011] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:31.435693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd036a-1cab-4c79-a319-544c77056b5e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.437242 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b5fd036a-1cab-4c79-a319-544c77056b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:31.445473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:31.448809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:31.500597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1019/2012] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:31.504948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5f79-cb14-4610-b0d1-546dc39afa96 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.512842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd036a-1cab-4c79-a319-544c77056b5e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.517704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f5f79-cb14-4610-b0d1-546dc39afa96 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.523460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562ed62-22e2-420c-bd6f-68d83825cae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.524044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562ed62-22e2-420c-bd6f-68d83825cae2 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.524182 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26dd83-be35-43ba-b7a7-c2ecea9841c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.524300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562ed62-22e2-420c-bd6f-68d83825cae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.524541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562ed62-22e2-420c-bd6f-68d83825cae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.524728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26dd83-be35-43ba-b7a7-c2ecea9841c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.525013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26dd83-be35-43ba-b7a7-c2ecea9841c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.525330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26dd83-be35-43ba-b7a7-c2ecea9841c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.526056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26dd83-be35-43ba-b7a7-c2ecea9841c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.528497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26dd83-be35-43ba-b7a7-c2ecea9841c9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.542939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562ed62-22e2-420c-bd6f-68d83825cae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.546952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562ed62-22e2-420c-bd6f-68d83825cae2 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.559038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 994/2013] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:31.577715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1020/2014] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:31 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:31.581948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562ed62-22e2-420c-bd6f-68d83825cae2 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.595529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562ed62-22e2-420c-bd6f-68d83825cae2 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.748657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8b60-4974-4417-ad6d-70b9de3beac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.749501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8b60-4974-4417-ad6d-70b9de3beac9 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.749778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8b60-4974-4417-ad6d-70b9de3beac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.750050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8b60-4974-4417-ad6d-70b9de3beac9 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.750708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a8b60-4974-4417-ad6d-70b9de3beac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.753238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a8b60-4974-4417-ad6d-70b9de3beac9 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.795206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 995/2015] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:31 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:31.799172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26dd83-be35-43ba-b7a7-c2ecea9841c9 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.826201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26dd83-be35-43ba-b7a7-c2ecea9841c9 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.842256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192e3db-8e3a-444d-b317-6b70f4db9a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.842884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192e3db-8e3a-444d-b317-6b70f4db9a13 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.843156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192e3db-8e3a-444d-b317-6b70f4db9a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.843424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192e3db-8e3a-444d-b317-6b70f4db9a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.844065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192e3db-8e3a-444d-b317-6b70f4db9a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.846119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192e3db-8e3a-444d-b317-6b70f4db9a13 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.862373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1021/2016] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:31.864301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8b60-4974-4417-ad6d-70b9de3beac9 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.878774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8b60-4974-4417-ad6d-70b9de3beac9 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:31.885186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:31.886002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:31.886347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:31.886682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] PATH_INFO: `/v3/projects/6453439bd05d4dc3ad137110440ca91f/users/f86aa339def24905b1a88e79f8a93158/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:31.907234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6453439bd05d4dc3ad137110440ca91f, user_id=f86aa339def24905b1a88e79f8a93158, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:31.909967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:31.932821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 996/2017] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:31 2022] PUT /identity/v3/projects/6453439bd05d4dc3ad137110440ca91f/users/f86aa339def24905b1a88e79f8a93158/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:31.936759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192e3db-8e3a-444d-b317-6b70f4db9a13 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:31.948300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192e3db-8e3a-444d-b317-6b70f4db9a13 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:32.008703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:32.010543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.014285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf30c-5b61-4fe3-98dc-ae3bcdd5c3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:32.015145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf30c-5b61-4fe3-98dc-ae3bcdd5c3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.015427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf30c-5b61-4fe3-98dc-ae3bcdd5c3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.015703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042cf30c-5b61-4fe3-98dc-ae3bcdd5c3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.016352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf30c-5b61-4fe3-98dc-ae3bcdd5c3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:32.018205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042cf30c-5b61-4fe3-98dc-ae3bcdd5c3e5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:32.027270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e994b8-929f-40d3-9ed4-b9f48e997e65 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:32.029707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1022/2018] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:32.031828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:32.032526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.032891 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.033233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] PATH_INFO: `/v3/projects/acb15e65b9704657a9a22347abeaa4fa` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.034495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb15e65b9704657a9a22347abeaa4fa)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:32.035399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf30c-5b61-4fe3-98dc-ae3bcdd5c3e5 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:32.036541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:32.054430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042cf30c-5b61-4fe3-98dc-ae3bcdd5c3e5 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:32.060104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1468f-0301-44db-b062-c3797b4d1e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:32.061079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1468f-0301-44db-b062-c3797b4d1e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.061506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1468f-0301-44db-b062-c3797b4d1e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.062022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1468f-0301-44db-b062-c3797b4d1e10 admin admin] PATH_INFO: `/v3/projects/6453439bd05d4dc3ad137110440ca91f/users/f86aa339def24905b1a88e79f8a93158/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.064147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1468f-0301-44db-b062-c3797b4d1e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6453439bd05d4dc3ad137110440ca91f, user_id=f86aa339def24905b1a88e79f8a93158, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:32.067843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1468f-0301-44db-b062-c3797b4d1e10 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:32.082453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1023/2019] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:32 2022] PUT /identity/v3/projects/6453439bd05d4dc3ad137110440ca91f/users/f86aa339def24905b1a88e79f8a93158/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:32.093391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22a08-42db-48e0-ba58-6fdb96541e2f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.094463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22a08-42db-48e0-ba58-6fdb96541e2f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.094771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22a08-42db-48e0-ba58-6fdb96541e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.121119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb15e65b9704657a9a22347abeaa4fa', 'request_id': 'req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.126751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-97d22a08-42db-48e0-ba58-6fdb96541e2f None None] MFA Rules not processed for user `f86aa339def24905b1a88e79f8a93158`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:32.127696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb15e65b9704657a9a22347abeaa4fa', 'request_id': 'req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.128770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-97d22a08-42db-48e0-ba58-6fdb96541e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.135270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb15e65b9704657a9a22347abeaa4fa', 'request_id': 'req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.139059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:32.162815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] The token cache is being invalidate because project acb15e65b9704657a9a22347abeaa4fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:32.162815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.162815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.162815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 997/2020] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:32 2022] DELETE /identity/v3/projects/acb15e65b9704657a9a22347abeaa4fa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:32.163552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:32.165006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.215103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1024/2021] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:32.226932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d22a08-42db-48e0-ba58-6fdb96541e2f None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:32.228288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-97d22a08-42db-48e0-ba58-6fdb96541e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.230960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b18b5-9fc8-4137-9d7b-d87b5df8f91e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:32.235601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:32.236227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.236505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.236776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] PATH_INFO: `/v3/users/62c9bf0592534c3ea0bf338d3360e80d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.238468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c9bf0592534c3ea0bf338d3360e80d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:32.240383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:32.242454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:32.290989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d22a08-42db-48e0-ba58-6fdb96541e2f None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:32.291802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c9bf0592534c3ea0bf338d3360e80d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.292471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c9bf0592534c3ea0bf338d3360e80d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.294648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:32.295263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.295690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.295840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.301073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c9bf0592534c3ea0bf338d3360e80d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.306129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:32.307486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:32.316586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c9bf0592534c3ea0bf338d3360e80d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.318943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1025/2022] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:32.324328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c9bf0592534c3ea0bf338d3360e80d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.330395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 998/2023] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:32 2022] DELETE /identity/v3/users/62c9bf0592534c3ea0bf338d3360e80d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:32.365686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:32.367506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.386285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f2b0-e5c8-49e0-92c4-c9d9360accba service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:32.451662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c6dd1-757b-4474-85b0-a942e9634ca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:32.452380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c6dd1-757b-4474-85b0-a942e9634ca1 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.452716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c6dd1-757b-4474-85b0-a942e9634ca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.453051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c6dd1-757b-4474-85b0-a942e9634ca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.455245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4c5c6dd1-757b-4474-85b0-a942e9634ca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.521934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c6dd1-757b-4474-85b0-a942e9634ca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:32.523364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c6dd1-757b-4474-85b0-a942e9634ca1 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:32.532443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1026/2024] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:32.627108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:32.628757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.711061 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1ddf4-11e0-415b-ba01-bdd6e22b5efc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:32.716168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:32.716840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.717148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.717428 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] PATH_INFO: `/v3/projects/b38d15340fe6487aa714f3378b617e2a` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.718675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38d15340fe6487aa714f3378b617e2a)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:32.720581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:32.740016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:32.781487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38d15340fe6487aa714f3378b617e2a', 'request_id': 'req-4efe865c-4d08-41b8-a50e-b2cd4137917d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.788134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38d15340fe6487aa714f3378b617e2a', 'request_id': 'req-4efe865c-4d08-41b8-a50e-b2cd4137917d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.795511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38d15340fe6487aa714f3378b617e2a', 'request_id': 'req-4efe865c-4d08-41b8-a50e-b2cd4137917d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.816163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] The token cache is being invalidate because project b38d15340fe6487aa714f3378b617e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:32.816752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.817859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4efe865c-4d08-41b8-a50e-b2cd4137917d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:32.820988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 999/2025] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:32 2022] DELETE /identity/v3/projects/b38d15340fe6487aa714f3378b617e2a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:32.825818 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb9e18-cc0b-4fcf-bc3f-724400661dbb None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.826271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb9e18-cc0b-4fcf-bc3f-724400661dbb None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.826598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbb9e18-cc0b-4fcf-bc3f-724400661dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.868447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-afbb9e18-cc0b-4fcf-bc3f-724400661dbb None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:32.869708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-afbb9e18-cc0b-4fcf-bc3f-724400661dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.949681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1027/2026] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:32.957983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176eb3d-9771-4e5d-b069-2e5820594c03 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:32.958297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176eb3d-9771-4e5d-b069-2e5820594c03 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:32.958545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176eb3d-9771-4e5d-b069-2e5820594c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:32.980391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a176eb3d-9771-4e5d-b069-2e5820594c03 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:32.981627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a176eb3d-9771-4e5d-b069-2e5820594c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:32.989643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1000/2027] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:32.993797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb9e18-cc0b-4fcf-bc3f-724400661dbb None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.003664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbb9e18-cc0b-4fcf-bc3f-724400661dbb None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.008198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.008791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.009090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.009374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.010014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:33.010316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.011913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.014241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:33.024188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1028/2028] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:33.028323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176eb3d-9771-4e5d-b069-2e5820594c03 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.037974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176eb3d-9771-4e5d-b069-2e5820594c03 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.042574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31653974-cc07-412f-add5-01a4614ccfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.043159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31653974-cc07-412f-add5-01a4614ccfbc admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.043439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31653974-cc07-412f-add5-01a4614ccfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.043707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31653974-cc07-412f-add5-01a4614ccfbc admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.044361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31653974-cc07-412f-add5-01a4614ccfbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.046268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31653974-cc07-412f-add5-01a4614ccfbc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.071712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1001/2029] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:33 2022] POST /identity/v3/projects => generated 407 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:33.075729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.086141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f6bc2-8142-49be-9e1e-edb7128c15b5 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.141279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195c038-36ea-481d-8bcd-e42533765681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.141886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195c038-36ea-481d-8bcd-e42533765681 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.142161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195c038-36ea-481d-8bcd-e42533765681 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.142425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195c038-36ea-481d-8bcd-e42533765681 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.143046 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195c038-36ea-481d-8bcd-e42533765681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.145643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195c038-36ea-481d-8bcd-e42533765681 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.174073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1029/2030] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:33 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:33.178217 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31653974-cc07-412f-add5-01a4614ccfbc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.188597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31653974-cc07-412f-add5-01a4614ccfbc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.194470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a7824-5113-420c-87af-6816600ea7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.195174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a7824-5113-420c-87af-6816600ea7fc admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.195515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a7824-5113-420c-87af-6816600ea7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.195792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a7824-5113-420c-87af-6816600ea7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.196423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a7824-5113-420c-87af-6816600ea7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.198021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a7824-5113-420c-87af-6816600ea7fc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.209408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1002/2031] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:33.213404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195c038-36ea-481d-8bcd-e42533765681 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.223979 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195c038-36ea-481d-8bcd-e42533765681 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.228945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2832e-82ee-4b7b-bbea-924d372134cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.229570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2832e-82ee-4b7b-bbea-924d372134cd admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.229847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2832e-82ee-4b7b-bbea-924d372134cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.230110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2832e-82ee-4b7b-bbea-924d372134cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.230740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2832e-82ee-4b7b-bbea-924d372134cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.232322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2832e-82ee-4b7b-bbea-924d372134cd admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.243319 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1030/2032] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:33.247877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a7824-5113-420c-87af-6816600ea7fc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.260169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a7824-5113-420c-87af-6816600ea7fc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.265575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe47a1-504b-4412-88c7-14ffd4f6ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.266252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe47a1-504b-4412-88c7-14ffd4f6ba8f admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.266535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe47a1-504b-4412-88c7-14ffd4f6ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.266840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe47a1-504b-4412-88c7-14ffd4f6ba8f admin admin] PATH_INFO: `/v3/projects/dec55f50470542ffbfd84b94acd17419/users/522d7bd22f2f48b9bd0c8d02945ebe04/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.283080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe47a1-504b-4412-88c7-14ffd4f6ba8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec55f50470542ffbfd84b94acd17419, user_id=522d7bd22f2f48b9bd0c8d02945ebe04, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.286637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe47a1-504b-4412-88c7-14ffd4f6ba8f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.300131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1003/2033] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:33 2022] PUT /identity/v3/projects/dec55f50470542ffbfd84b94acd17419/users/522d7bd22f2f48b9bd0c8d02945ebe04/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:33.312135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bbbf0-e85e-4686-9a7f-f07a026dda64 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.312422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bbbf0-e85e-4686-9a7f-f07a026dda64 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.312788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bbbf0-e85e-4686-9a7f-f07a026dda64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.341944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-995bbbf0-e85e-4686-9a7f-f07a026dda64 None None] MFA Rules not processed for user `522d7bd22f2f48b9bd0c8d02945ebe04`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:33.343132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-995bbbf0-e85e-4686-9a7f-f07a026dda64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:33.407612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1031/2034] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:33 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:33.412238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe47a1-504b-4412-88c7-14ffd4f6ba8f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.474078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe47a1-504b-4412-88c7-14ffd4f6ba8f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.478934 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf95d18-673f-418d-9251-038c2a3d3668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.479571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf95d18-673f-418d-9251-038c2a3d3668 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.479869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf95d18-673f-418d-9251-038c2a3d3668 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.480136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf95d18-673f-418d-9251-038c2a3d3668 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.480783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf95d18-673f-418d-9251-038c2a3d3668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.482945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf95d18-673f-418d-9251-038c2a3d3668 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.509882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1004/2035] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:33 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:33.513933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bbbf0-e85e-4686-9a7f-f07a026dda64 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.525106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bbbf0-e85e-4686-9a7f-f07a026dda64 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.579149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227189eb-7963-4523-903f-e97f17137df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.579667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227189eb-7963-4523-903f-e97f17137df6 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.579942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227189eb-7963-4523-903f-e97f17137df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.580209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227189eb-7963-4523-903f-e97f17137df6 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.580826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227189eb-7963-4523-903f-e97f17137df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.585283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227189eb-7963-4523-903f-e97f17137df6 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.595175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:33.607224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1032/2036] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:33 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:33.611211 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf95d18-673f-418d-9251-038c2a3d3668 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.621298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf95d18-673f-418d-9251-038c2a3d3668 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.626062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411e6b4-1e05-4b01-9b83-27d571f14b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.626648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411e6b4-1e05-4b01-9b83-27d571f14b6e admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.626922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411e6b4-1e05-4b01-9b83-27d571f14b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.627191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6411e6b4-1e05-4b01-9b83-27d571f14b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.627819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411e6b4-1e05-4b01-9b83-27d571f14b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.629932 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6411e6b4-1e05-4b01-9b83-27d571f14b6e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.640248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1005/2037] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:33.644964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227189eb-7963-4523-903f-e97f17137df6 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.654886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227189eb-7963-4523-903f-e97f17137df6 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.659588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a0abf-d339-47a0-a4cc-626275dd468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.660225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a0abf-d339-47a0-a4cc-626275dd468e admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.660497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a0abf-d339-47a0-a4cc-626275dd468e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.660805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a0abf-d339-47a0-a4cc-626275dd468e admin admin] PATH_INFO: `/v3/projects/e5fd1d7a457449859f4b9f7c3b297b79/users/bb0b6c7db1b54542b7e37c24bed9d09b/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.675056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a0abf-d339-47a0-a4cc-626275dd468e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5fd1d7a457449859f4b9f7c3b297b79, user_id=bb0b6c7db1b54542b7e37c24bed9d09b, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.677713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a0abf-d339-47a0-a4cc-626275dd468e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.690448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1033/2038] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:33 2022] PUT /identity/v3/projects/e5fd1d7a457449859f4b9f7c3b297b79/users/bb0b6c7db1b54542b7e37c24bed9d09b/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:33.694995 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411e6b4-1e05-4b01-9b83-27d571f14b6e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.705023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6411e6b4-1e05-4b01-9b83-27d571f14b6e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.717568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:33.764221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a22708-ffb2-495b-9d18-6e3c48c4589c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.764833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22708-ffb2-495b-9d18-6e3c48c4589c admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.765134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22708-ffb2-495b-9d18-6e3c48c4589c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.765408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a22708-ffb2-495b-9d18-6e3c48c4589c admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.766042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a22708-ffb2-495b-9d18-6e3c48c4589c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.767610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a22708-ffb2-495b-9d18-6e3c48c4589c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.777052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1006/2039] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:33.781391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a0abf-d339-47a0-a4cc-626275dd468e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.791433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a0abf-d339-47a0-a4cc-626275dd468e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:33.796138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:33.796780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.797052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.797358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] PATH_INFO: `/v3/projects/e5fd1d7a457449859f4b9f7c3b297b79/users/bb0b6c7db1b54542b7e37c24bed9d09b/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.799223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5fd1d7a457449859f4b9f7c3b297b79, user_id=bb0b6c7db1b54542b7e37c24bed9d09b, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:33.802398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:33.812510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1034/2040] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:33 2022] PUT /identity/v3/projects/e5fd1d7a457449859f4b9f7c3b297b79/users/bb0b6c7db1b54542b7e37c24bed9d09b/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:33.823040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731da73-d0d9-4838-9a02-d8ff3e31e8e2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:33.823378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731da73-d0d9-4838-9a02-d8ff3e31e8e2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:33.823652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731da73-d0d9-4838-9a02-d8ff3e31e8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:33.852738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-2731da73-d0d9-4838-9a02-d8ff3e31e8e2 None None] MFA Rules not processed for user `bb0b6c7db1b54542b7e37c24bed9d09b`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:33.854146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2731da73-d0d9-4838-9a02-d8ff3e31e8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:33.936741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1007/2041] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:33.948099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:33.949842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:34.025883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ec06b-92d0-45ae-84d3-bd1f2b3c1926 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:34.032061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:34.032061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:34.032423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:34.032795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:34.046935 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:34.049133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:34.064588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1035/2042] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:33 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:34.304248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731da73-d0d9-4838-9a02-d8ff3e31e8e2 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:34.315022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731da73-d0d9-4838-9a02-d8ff3e31e8e2 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:34.319178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abd957-3062-4b51-9253-97f29430f30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:34.319832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abd957-3062-4b51-9253-97f29430f30b service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:34.320114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abd957-3062-4b51-9253-97f29430f30b service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:34.320473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abd957-3062-4b51-9253-97f29430f30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:34.416378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abd957-3062-4b51-9253-97f29430f30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:34.417953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abd957-3062-4b51-9253-97f29430f30b service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:34.428913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1008/2043] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:34 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:35.973840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:35.975234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:36.039294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f297041-417f-4c87-b3d6-4566e36afb8b service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:36.044164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:36.044786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:36.045092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:36.045355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] PATH_INFO: `/v3/users/7e6b6288f32048ca8cc518463f4520b5` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:36.046457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6b6288f32048ca8cc518463f4520b5)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:36.048398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:36.092792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6b6288f32048ca8cc518463f4520b5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.093462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6b6288f32048ca8cc518463f4520b5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.104041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:36.106439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6b6288f32048ca8cc518463f4520b5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.114675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6b6288f32048ca8cc518463f4520b5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.115814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6b6288f32048ca8cc518463f4520b5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.128766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1036/2044] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:35 2022] DELETE /identity/v3/users/7e6b6288f32048ca8cc518463f4520b5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:36.133303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b1817-b5cd-4e1c-97cb-eeff541f21a5 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:36.133756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b1817-b5cd-4e1c-97cb-eeff541f21a5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:36.133843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b1817-b5cd-4e1c-97cb-eeff541f21a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:36.153182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-bc2b1817-b5cd-4e1c-97cb-eeff541f21a5 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:36.154380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bc2b1817-b5cd-4e1c-97cb-eeff541f21a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:36.183587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:36.212462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1009/2045] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:36.223052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:36.224399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:36.290103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29ae76-0ad4-418c-93f0-36b064e1b0d9 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:36.293849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f70f-4975-4f5c-9eae-167580c120ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:36.294464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f70f-4975-4f5c-9eae-167580c120ab service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:36.294741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f70f-4975-4f5c-9eae-167580c120ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:36.295014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48f70f-4975-4f5c-9eae-167580c120ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:36.305054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48f70f-4975-4f5c-9eae-167580c120ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:36.306527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48f70f-4975-4f5c-9eae-167580c120ab service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:36.317394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1037/2046] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:36.621966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b1817-b5cd-4e1c-97cb-eeff541f21a5 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:36.624504 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bc2b1817-b5cd-4e1c-97cb-eeff541f21a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:36.646035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b1817-b5cd-4e1c-97cb-eeff541f21a5 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:36.650719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:36.651375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:36.651691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:36.651993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] PATH_INFO: `/v3/projects/86274b6395e041e79c291d3ffbe904c8` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:36.653153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] RBAC: Authorizing `identity:delete_project(project_id=86274b6395e041e79c291d3ffbe904c8)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:36.655050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:36.703124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86274b6395e041e79c291d3ffbe904c8', 'request_id': 'req-e0e536d2-b5b7-443c-87e5-484d2613846d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.709329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86274b6395e041e79c291d3ffbe904c8', 'request_id': 'req-e0e536d2-b5b7-443c-87e5-484d2613846d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.715411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86274b6395e041e79c291d3ffbe904c8', 'request_id': 'req-e0e536d2-b5b7-443c-87e5-484d2613846d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.727539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] The token cache is being invalidate because project 86274b6395e041e79c291d3ffbe904c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:36.727763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.728386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.729968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1010/2047] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:36 2022] DELETE /identity/v3/projects/86274b6395e041e79c291d3ffbe904c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:36.734064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f70f-4975-4f5c-9eae-167580c120ab service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:36.744357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48f70f-4975-4f5c-9eae-167580c120ab service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:36.804085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:36.804736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:36.805021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:36.805481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] PATH_INFO: `/v3/users/f86aa339def24905b1a88e79f8a93158` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:36.806982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86aa339def24905b1a88e79f8a93158)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:36.809580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:36.860390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86aa339def24905b1a88e79f8a93158'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.861055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86aa339def24905b1a88e79f8a93158'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.873729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86aa339def24905b1a88e79f8a93158'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.882359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86aa339def24905b1a88e79f8a93158'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.882983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-273ec943-99d0-4278-858f-2bf31ed91b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86aa339def24905b1a88e79f8a93158'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:36.891823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:36.896783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1038/2048] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:36 2022] DELETE /identity/v3/users/f86aa339def24905b1a88e79f8a93158 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:37.169156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:37.170700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:37.185676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e536d2-b5b7-443c-87e5-484d2613846d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:37.243583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:37.244118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.244399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.244670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] PATH_INFO: `/v3/projects/6453439bd05d4dc3ad137110440ca91f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.245806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6453439bd05d4dc3ad137110440ca91f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:37.247948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:37.302151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6453439bd05d4dc3ad137110440ca91f', 'request_id': 'req-0b12d378-49a8-40fa-8bc3-2965555e50b4'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:37.308649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6453439bd05d4dc3ad137110440ca91f', 'request_id': 'req-0b12d378-49a8-40fa-8bc3-2965555e50b4'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:37.312340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:37.318926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6453439bd05d4dc3ad137110440ca91f', 'request_id': 'req-0b12d378-49a8-40fa-8bc3-2965555e50b4'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:37.331248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] The token cache is being invalidate because project 6453439bd05d4dc3ad137110440ca91f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:37.331538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:37.332169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0b12d378-49a8-40fa-8bc3-2965555e50b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:37.333686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1011/2049] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:37 2022] DELETE /identity/v3/projects/6453439bd05d4dc3ad137110440ca91f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:37.338518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f38639-1f3b-427e-813e-bd3b71f47f8b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.338978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f38639-1f3b-427e-813e-bd3b71f47f8b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.339193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f38639-1f3b-427e-813e-bd3b71f47f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.359257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-86f38639-1f3b-427e-813e-bd3b71f47f8b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:37.360449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-86f38639-1f3b-427e-813e-bd3b71f47f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:37.418458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1039/2050] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:37.426736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22d7-0b27-467d-ad87-0bbd38ddfd1f None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.427018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22d7-0b27-467d-ad87-0bbd38ddfd1f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.427291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b22d7-0b27-467d-ad87-0bbd38ddfd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.446461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a02b22d7-0b27-467d-ad87-0bbd38ddfd1f None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:37.447840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a02b22d7-0b27-467d-ad87-0bbd38ddfd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:37.455665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1012/2051] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:37.460481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f38639-1f3b-427e-813e-bd3b71f47f8b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:37.469977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f38639-1f3b-427e-813e-bd3b71f47f8b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:37.475386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:37.475989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.476265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.476534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.477208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:37.477509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:37.479089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:37.490926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1040/2052] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:37.498605 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b22d7-0b27-467d-ad87-0bbd38ddfd1f None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:37.507631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b22d7-0b27-467d-ad87-0bbd38ddfd1f None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:37.512241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27a7ef-7f3f-49fd-b7e8-9bc1dd22ff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:37.512833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27a7ef-7f3f-49fd-b7e8-9bc1dd22ff0b admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.513138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27a7ef-7f3f-49fd-b7e8-9bc1dd22ff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.513412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27a7ef-7f3f-49fd-b7e8-9bc1dd22ff0b admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.514031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27a7ef-7f3f-49fd-b7e8-9bc1dd22ff0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:37.515921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27a7ef-7f3f-49fd-b7e8-9bc1dd22ff0b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:37.550028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1013/2053] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:37 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:37.554396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:37.569313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df0bce-eac0-4ca3-bdf9-bd3e9b20db6d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:37.629915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a50712-13be-429c-b2ed-b547aad843ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:37.630784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50712-13be-429c-b2ed-b547aad843ff admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.631133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50712-13be-429c-b2ed-b547aad843ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.631386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a50712-13be-429c-b2ed-b547aad843ff admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.632207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a50712-13be-429c-b2ed-b547aad843ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:37.634480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a50712-13be-429c-b2ed-b547aad843ff admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:37.681261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1041/2054] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:37 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:37.682554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27a7ef-7f3f-49fd-b7e8-9bc1dd22ff0b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:37.710602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27a7ef-7f3f-49fd-b7e8-9bc1dd22ff0b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:37.717085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b661d2-7a25-4d10-88ca-3766e0222a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:37.717799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b661d2-7a25-4d10-88ca-3766e0222a25 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.718102 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b661d2-7a25-4d10-88ca-3766e0222a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.718367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b661d2-7a25-4d10-88ca-3766e0222a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.719142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b661d2-7a25-4d10-88ca-3766e0222a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:37.721127 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b661d2-7a25-4d10-88ca-3766e0222a25 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:37.745121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1014/2055] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:37.748013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a50712-13be-429c-b2ed-b547aad843ff admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:37.767013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a50712-13be-429c-b2ed-b547aad843ff admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:37.775292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdab8d0-5d7c-4297-b6d3-61752e32bf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:37.776550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdab8d0-5d7c-4297-b6d3-61752e32bf10 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.776784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdab8d0-5d7c-4297-b6d3-61752e32bf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.777178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdab8d0-5d7c-4297-b6d3-61752e32bf10 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.777833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdab8d0-5d7c-4297-b6d3-61752e32bf10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:37.779676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdab8d0-5d7c-4297-b6d3-61752e32bf10 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:37.804435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1042/2056] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:37.804592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b661d2-7a25-4d10-88ca-3766e0222a25 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:37.811128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b661d2-7a25-4d10-88ca-3766e0222a25 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:37.816632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281035a5-4b3e-4cd0-984a-d3814bf29196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:37.817739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281035a5-4b3e-4cd0-984a-d3814bf29196 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.818027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281035a5-4b3e-4cd0-984a-d3814bf29196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.818353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281035a5-4b3e-4cd0-984a-d3814bf29196 admin admin] PATH_INFO: `/v3/projects/0726aaf28efa436a96af3484733c09d1/users/bf4d8d23edbc46c29d2f7720284e6d72/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.835008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281035a5-4b3e-4cd0-984a-d3814bf29196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0726aaf28efa436a96af3484733c09d1, user_id=bf4d8d23edbc46c29d2f7720284e6d72, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:37.838411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281035a5-4b3e-4cd0-984a-d3814bf29196 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:37.856677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1015/2057] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:37 2022] PUT /identity/v3/projects/0726aaf28efa436a96af3484733c09d1/users/bf4d8d23edbc46c29d2f7720284e6d72/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:37.867981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13bfe3-c62a-4835-94ba-185b9f327922 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:37.868258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13bfe3-c62a-4835-94ba-185b9f327922 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:37.868542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13bfe3-c62a-4835-94ba-185b9f327922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:37.900636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1d13bfe3-c62a-4835-94ba-185b9f327922 None None] MFA Rules not processed for user `bf4d8d23edbc46c29d2f7720284e6d72`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:37.902016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1d13bfe3-c62a-4835-94ba-185b9f327922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:37.979842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:37.998841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1043/2058] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:37.999312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281035a5-4b3e-4cd0-984a-d3814bf29196 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:38.075826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281035a5-4b3e-4cd0-984a-d3814bf29196 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:38.082844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529c7c-1328-4497-9dbf-637eb3b0ef9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:38.083564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529c7c-1328-4497-9dbf-637eb3b0ef9b admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:38.083944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529c7c-1328-4497-9dbf-637eb3b0ef9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:38.084350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f529c7c-1328-4497-9dbf-637eb3b0ef9b admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:38.085165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f529c7c-1328-4497-9dbf-637eb3b0ef9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:38.088649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f529c7c-1328-4497-9dbf-637eb3b0ef9b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:38.123935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1016/2059] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:37 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:38.132115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13bfe3-c62a-4835-94ba-185b9f327922 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:38.146346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13bfe3-c62a-4835-94ba-185b9f327922 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:38.234456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:38.235075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:38.235358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:38.235632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:38.236276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:38.238759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:38.265986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1044/2060] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:38 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:38.270378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529c7c-1328-4497-9dbf-637eb3b0ef9b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:38.282253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f529c7c-1328-4497-9dbf-637eb3b0ef9b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:38.288173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89d3cc-3c14-45f2-85a8-954851ddb171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:38.289353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89d3cc-3c14-45f2-85a8-954851ddb171 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:38.289692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89d3cc-3c14-45f2-85a8-954851ddb171 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:38.290108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89d3cc-3c14-45f2-85a8-954851ddb171 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:38.291410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89d3cc-3c14-45f2-85a8-954851ddb171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:38.293609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89d3cc-3c14-45f2-85a8-954851ddb171 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:38.334625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:38.336570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:38.337611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1017/2061] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:38.341774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89d3cc-3c14-45f2-85a8-954851ddb171 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:38.352268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89d3cc-3c14-45f2-85a8-954851ddb171 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:38.358763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ea69-5b83-4fd1-b9d7-e292fb987963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:38.359513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ea69-5b83-4fd1-b9d7-e292fb987963 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:38.359870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ea69-5b83-4fd1-b9d7-e292fb987963 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:38.360212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ea69-5b83-4fd1-b9d7-e292fb987963 admin admin] PATH_INFO: `/v3/projects/e1e0f21d20e8437f8d4f2d8b8daae53e/users/441d4075f62b43ae81eb4be48979bdad/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:38.377045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99ea69-5b83-4fd1-b9d7-e292fb987963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e0f21d20e8437f8d4f2d8b8daae53e, user_id=441d4075f62b43ae81eb4be48979bdad, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:38.381417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99ea69-5b83-4fd1-b9d7-e292fb987963 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:38.403761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1018/2062] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:38 2022] PUT /identity/v3/projects/e1e0f21d20e8437f8d4f2d8b8daae53e/users/441d4075f62b43ae81eb4be48979bdad/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:38.405171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a537af-1246-4b48-a6c0-22a31f52ea6d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:38.407768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ea69-5b83-4fd1-b9d7-e292fb987963 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:38.451565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:38.477473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:38.478495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:38.479161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:38.479472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:38.481696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:38.482229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ea69-5b83-4fd1-b9d7-e292fb987963 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:38.487890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dd688-bc75-478b-a374-b735f10929c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:38.488673 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dd688-bc75-478b-a374-b735f10929c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:38.488940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dd688-bc75-478b-a374-b735f10929c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:38.500089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25dd688-bc75-478b-a374-b735f10929c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:38.500089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dd688-bc75-478b-a374-b735f10929c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:38.507803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25dd688-bc75-478b-a374-b735f10929c3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:38.529126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1019/2063] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:38 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:38.534064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dd688-bc75-478b-a374-b735f10929c3 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:38.555584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25dd688-bc75-478b-a374-b735f10929c3 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:38.576505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528798-d8be-4c55-a5ac-0ae895bb17c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:38.579152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528798-d8be-4c55-a5ac-0ae895bb17c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:38.579572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528798-d8be-4c55-a5ac-0ae895bb17c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:38.579962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528798-d8be-4c55-a5ac-0ae895bb17c5 admin admin] PATH_INFO: `/v3/projects/e1e0f21d20e8437f8d4f2d8b8daae53e/users/441d4075f62b43ae81eb4be48979bdad/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:38.583272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528798-d8be-4c55-a5ac-0ae895bb17c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e0f21d20e8437f8d4f2d8b8daae53e, user_id=441d4075f62b43ae81eb4be48979bdad, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:38.585809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:38.587651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528798-d8be-4c55-a5ac-0ae895bb17c5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:38.601387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:38.602944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:38.649161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1020/2064] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:38 2022] PUT /identity/v3/projects/e1e0f21d20e8437f8d4f2d8b8daae53e/users/441d4075f62b43ae81eb4be48979bdad/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:38.656308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168a8d-bd0d-459e-b862-890af0ea4dc2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:38.656539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168a8d-bd0d-459e-b862-890af0ea4dc2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:38.656991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168a8d-bd0d-459e-b862-890af0ea4dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:38.658301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1045/2065] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:38.723078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-47168a8d-bd0d-459e-b862-890af0ea4dc2 None None] MFA Rules not processed for user `441d4075f62b43ae81eb4be48979bdad`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:38.724646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-47168a8d-bd0d-459e-b862-890af0ea4dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:38.907599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1021/2066] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:39.027847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:39.029810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:39.048406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619dcb60-3ff3-470e-b14e-2773d206dc93 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:39.120019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e42ec7-39c4-499b-82f4-e1b84104349a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:39.120635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e42ec7-39c4-499b-82f4-e1b84104349a service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:39.120920 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e42ec7-39c4-499b-82f4-e1b84104349a service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:39.121222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e42ec7-39c4-499b-82f4-e1b84104349a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:39.131425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e42ec7-39c4-499b-82f4-e1b84104349a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:39.132841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e42ec7-39c4-499b-82f4-e1b84104349a service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:39.143202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1046/2067] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:39.766316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168a8d-bd0d-459e-b862-890af0ea4dc2 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:39.776630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168a8d-bd0d-459e-b862-890af0ea4dc2 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:39.780676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:39.781329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:39.781803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:39.782085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:39.784200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:39.803542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e42ec7-39c4-499b-82f4-e1b84104349a service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:39.812990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e42ec7-39c4-499b-82f4-e1b84104349a service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:39.818535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441d4075f62b43ae81eb4be48979bdad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1e0f21d20e8437f8d4f2d8b8daae53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:39.819124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:39.819403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:39.819673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:39.820305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:39.820612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:39.822228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:39.834280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1047/2068] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:49:39.838414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:39.847801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf51f59-4c95-4a07-90a7-1b7bbbf908ed tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:39.851420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:39.852785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ea5fd-9869-497e-b761-a86145afcae6 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441d4075f62b43ae81eb4be48979bdad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1e0f21d20e8437f8d4f2d8b8daae53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:39.852917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:39.853386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea5fd-9869-497e-b761-a86145afcae6 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:39.853669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea5fd-9869-497e-b761-a86145afcae6 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:39.853943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ea5fd-9869-497e-b761-a86145afcae6 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:39.854561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ea5fd-9869-497e-b761-a86145afcae6 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:39.856422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ea5fd-9869-497e-b761-a86145afcae6 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:39.863187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1022/2069] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:39.887101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1048/2070] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:39 2022] POST /identity/v3/projects => generated 354 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:39.891285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:39.902600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941ba19-dbe2-448b-86a9-110aba782a92 service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:39.939867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:39.967348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441d4075f62b43ae81eb4be48979bdad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1e0f21d20e8437f8d4f2d8b8daae53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:39.967971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:39.968258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:39.968534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:39.969225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:39.969538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:39.971144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:39.982770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1023/2071] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:40.020503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ea5fd-9869-497e-b761-a86145afcae6 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:40.030822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ea5fd-9869-497e-b761-a86145afcae6 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:40.035450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '441d4075f62b43ae81eb4be48979bdad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1e0f21d20e8437f8d4f2d8b8daae53e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:40.036072 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:40.036348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:40.036620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] PATH_INFO: `/v3/projects/5aa264f17ee64c5fbe4dfeb657c12e57` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:40.037749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5aa264f17ee64c5fbe4dfeb657c12e57)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:40.039638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:40.086889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5aa264f17ee64c5fbe4dfeb657c12e57', 'request_id': 'req-96e678f2-6ee7-4c77-85b1-35347f67ba79'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.093316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5aa264f17ee64c5fbe4dfeb657c12e57', 'request_id': 'req-96e678f2-6ee7-4c77-85b1-35347f67ba79'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.100523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5aa264f17ee64c5fbe4dfeb657c12e57', 'request_id': 'req-96e678f2-6ee7-4c77-85b1-35347f67ba79'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.113489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] The token cache is being invalidate because project 5aa264f17ee64c5fbe4dfeb657c12e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:40.113776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.114416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-96e678f2-6ee7-4c77-85b1-35347f67ba79 tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.115948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1049/2072] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:40 2022] DELETE /identity/v3/projects/5aa264f17ee64c5fbe4dfeb657c12e57 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:40.297813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:40.299485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:40.367019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5f609-ebd7-4676-8ad3-e09f9df70d4b tempest-VolumeQuotaClassesTest-2041587276 tempest-VolumeQuotaClassesTest-2041587276-project-admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:40.422209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:40.422843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:40.423154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:40.423430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] PATH_INFO: `/v3/users/bf4d8d23edbc46c29d2f7720284e6d72` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:40.424548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4d8d23edbc46c29d2f7720284e6d72)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:40.427026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:40.472745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4d8d23edbc46c29d2f7720284e6d72'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.473461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4d8d23edbc46c29d2f7720284e6d72'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.484367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4d8d23edbc46c29d2f7720284e6d72'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.500783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4d8d23edbc46c29d2f7720284e6d72'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.514870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4d8d23edbc46c29d2f7720284e6d72'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:40.516699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:40.529007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1024/2073] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:40 2022] DELETE /identity/v3/users/bf4d8d23edbc46c29d2f7720284e6d72 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:40.533690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a22c-1859-4263-b4a0-df5a26859ddd None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:40.534552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a22c-1859-4263-b4a0-df5a26859ddd None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:40.534838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a22c-1859-4263-b4a0-df5a26859ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:40.556435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ece7a22c-1859-4263-b4a0-df5a26859ddd None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:40.557817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ece7a22c-1859-4263-b4a0-df5a26859ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:40.602207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:40.621823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1050/2074] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:40.631418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:40.633104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:40.695832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a0293-c267-4d32-8f2f-adf6db0a0739 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:40.699471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a37e0e-e81c-4ca7-8d71-4e82b71d4d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:40.700074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a37e0e-e81c-4ca7-8d71-4e82b71d4d9e service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:40.700350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a37e0e-e81c-4ca7-8d71-4e82b71d4d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:40.700650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a37e0e-e81c-4ca7-8d71-4e82b71d4d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:40.710246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a37e0e-e81c-4ca7-8d71-4e82b71d4d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:40.711699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a37e0e-e81c-4ca7-8d71-4e82b71d4d9e service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:40.721360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1025/2075] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:41.001647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7a22c-1859-4263-b4a0-df5a26859ddd None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:41.003645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ece7a22c-1859-4263-b4a0-df5a26859ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:41.017009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7a22c-1859-4263-b4a0-df5a26859ddd None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:41.023106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:41.023868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:41.024242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:41.024622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] PATH_INFO: `/v3/projects/0726aaf28efa436a96af3484733c09d1` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:41.025993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0726aaf28efa436a96af3484733c09d1)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:41.028064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:41.079099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0726aaf28efa436a96af3484733c09d1', 'request_id': 'req-a85b977f-499a-429c-a531-3d2d0f4bb019'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.085817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0726aaf28efa436a96af3484733c09d1', 'request_id': 'req-a85b977f-499a-429c-a531-3d2d0f4bb019'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.092138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0726aaf28efa436a96af3484733c09d1', 'request_id': 'req-a85b977f-499a-429c-a531-3d2d0f4bb019'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.104804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] The token cache is being invalidate because project 0726aaf28efa436a96af3484733c09d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:41.105118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.105747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.107215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1051/2076] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:41 2022] DELETE /identity/v3/projects/0726aaf28efa436a96af3484733c09d1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:41.111159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a37e0e-e81c-4ca7-8d71-4e82b71d4d9e service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:41.120739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a37e0e-e81c-4ca7-8d71-4e82b71d4d9e service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:41.179550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:41.180170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:41.180447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:41.180714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] PATH_INFO: `/v3/users/441d4075f62b43ae81eb4be48979bdad` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:41.181835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=441d4075f62b43ae81eb4be48979bdad)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:41.183728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:41.229216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '441d4075f62b43ae81eb4be48979bdad'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.229863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '441d4075f62b43ae81eb4be48979bdad'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.236810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '441d4075f62b43ae81eb4be48979bdad'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.250699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '441d4075f62b43ae81eb4be48979bdad'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.258493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-74e7db25-552f-40e0-95e1-a26ca0715615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '441d4075f62b43ae81eb4be48979bdad'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.264554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1026/2077] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:41 2022] DELETE /identity/v3/users/441d4075f62b43ae81eb4be48979bdad => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:41.533579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:41.535183 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:41.551197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b977f-499a-429c-a531-3d2d0f4bb019 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:41.617166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:41.617803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:41.618081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:41.618350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] PATH_INFO: `/v3/projects/e1e0f21d20e8437f8d4f2d8b8daae53e` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:41.619455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e0f21d20e8437f8d4f2d8b8daae53e)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:41.621412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:41.667848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e0f21d20e8437f8d4f2d8b8daae53e', 'request_id': 'req-519375f2-e7fc-48c7-b742-9ad14b30d022'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.674147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e0f21d20e8437f8d4f2d8b8daae53e', 'request_id': 'req-519375f2-e7fc-48c7-b742-9ad14b30d022'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.680356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e0f21d20e8437f8d4f2d8b8daae53e', 'request_id': 'req-519375f2-e7fc-48c7-b742-9ad14b30d022'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.695148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:41.699436 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] The token cache is being invalidate because project e1e0f21d20e8437f8d4f2d8b8daae53e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:41.700150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.701493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-519375f2-e7fc-48c7-b742-9ad14b30d022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:41.704090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1052/2078] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:41 2022] DELETE /identity/v3/projects/e1e0f21d20e8437f8d4f2d8b8daae53e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:41.709136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac05ca-2715-47d2-8c0f-de02e170233f None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:41.709524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac05ca-2715-47d2-8c0f-de02e170233f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:41.710078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac05ca-2715-47d2-8c0f-de02e170233f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:41.730294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f2ac05ca-2715-47d2-8c0f-de02e170233f None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:41.731487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f2ac05ca-2715-47d2-8c0f-de02e170233f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:41.745600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:41.802336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1027/2079] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:41.807492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ba489-faac-441a-b9aa-dfbe0fe27d72 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:41.807779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ba489-faac-441a-b9aa-dfbe0fe27d72 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:41.808073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ba489-faac-441a-b9aa-dfbe0fe27d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:41.851914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-880ba489-faac-441a-b9aa-dfbe0fe27d72 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:41.853145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-880ba489-faac-441a-b9aa-dfbe0fe27d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:41.864922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1053/2080] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:41.866046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac05ca-2715-47d2-8c0f-de02e170233f None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:41.877477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac05ca-2715-47d2-8c0f-de02e170233f None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:41.883011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:41.883614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:41.883890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:41.884158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:41.884801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:41.885125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:41.886710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:41.905491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1028/2081] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:41.909312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ba489-faac-441a-b9aa-dfbe0fe27d72 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:41.918837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ba489-faac-441a-b9aa-dfbe0fe27d72 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:41.923585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574d63d-d921-4d37-9bd2-44b1eab61736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:41.924185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574d63d-d921-4d37-9bd2-44b1eab61736 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:41.924550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574d63d-d921-4d37-9bd2-44b1eab61736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:41.924824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574d63d-d921-4d37-9bd2-44b1eab61736 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:41.925487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574d63d-d921-4d37-9bd2-44b1eab61736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:41.927377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574d63d-d921-4d37-9bd2-44b1eab61736 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:41.957639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1054/2082] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:41 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:41.961971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:41.972418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18ce22-1d3b-424d-b08c-5a0e57b5a357 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.045533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826200ad-29f4-4be2-8451-0a365efc8958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.046043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826200ad-29f4-4be2-8451-0a365efc8958 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.046323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826200ad-29f4-4be2-8451-0a365efc8958 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.046591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826200ad-29f4-4be2-8451-0a365efc8958 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.047210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826200ad-29f4-4be2-8451-0a365efc8958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.049235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826200ad-29f4-4be2-8451-0a365efc8958 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.072862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1029/2083] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:41 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:42.076992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574d63d-d921-4d37-9bd2-44b1eab61736 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.091566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574d63d-d921-4d37-9bd2-44b1eab61736 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.097520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0331431-d60e-4291-b64a-cab594157b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.098124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331431-d60e-4291-b64a-cab594157b54 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.098403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331431-d60e-4291-b64a-cab594157b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.098675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331431-d60e-4291-b64a-cab594157b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.099311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0331431-d60e-4291-b64a-cab594157b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.100880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0331431-d60e-4291-b64a-cab594157b54 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.111292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1055/2084] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:42.116016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826200ad-29f4-4be2-8451-0a365efc8958 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.126540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826200ad-29f4-4be2-8451-0a365efc8958 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.132159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f073-c6b4-4deb-86a6-111078318f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.132955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f073-c6b4-4deb-86a6-111078318f6e admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.133445 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f073-c6b4-4deb-86a6-111078318f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.133721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f073-c6b4-4deb-86a6-111078318f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.134436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f073-c6b4-4deb-86a6-111078318f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.136097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f073-c6b4-4deb-86a6-111078318f6e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.147272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1030/2085] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:42.152055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0331431-d60e-4291-b64a-cab594157b54 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.162907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0331431-d60e-4291-b64a-cab594157b54 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.168424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869a487-47e9-45fc-ba5c-e1dd6eccaa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.169178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a487-47e9-45fc-ba5c-e1dd6eccaa16 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.169460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a487-47e9-45fc-ba5c-e1dd6eccaa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.169732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a487-47e9-45fc-ba5c-e1dd6eccaa16 admin admin] PATH_INFO: `/v3/projects/930d423c40374d72ad0b4fcfcb22221c/users/ef784188fe684305817ed12f41bd6cdd/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.186304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869a487-47e9-45fc-ba5c-e1dd6eccaa16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=930d423c40374d72ad0b4fcfcb22221c, user_id=ef784188fe684305817ed12f41bd6cdd, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.189003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869a487-47e9-45fc-ba5c-e1dd6eccaa16 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.202437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1056/2086] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:42 2022] PUT /identity/v3/projects/930d423c40374d72ad0b4fcfcb22221c/users/ef784188fe684305817ed12f41bd6cdd/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:42.212927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ed920-a43f-4d17-9514-5be9a435eeef None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.213247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ed920-a43f-4d17-9514-5be9a435eeef None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.213520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ed920-a43f-4d17-9514-5be9a435eeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.240645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-484ed920-a43f-4d17-9514-5be9a435eeef None None] MFA Rules not processed for user `ef784188fe684305817ed12f41bd6cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:42.241901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-484ed920-a43f-4d17-9514-5be9a435eeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:42.306876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1031/2087] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:42.311702 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869a487-47e9-45fc-ba5c-e1dd6eccaa16 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.378668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869a487-47e9-45fc-ba5c-e1dd6eccaa16 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.383405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20480681-15fa-428a-9991-94e6b7c7bb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.384010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20480681-15fa-428a-9991-94e6b7c7bb93 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.384290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20480681-15fa-428a-9991-94e6b7c7bb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.384563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20480681-15fa-428a-9991-94e6b7c7bb93 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.385252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20480681-15fa-428a-9991-94e6b7c7bb93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.387462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20480681-15fa-428a-9991-94e6b7c7bb93 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.420005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1057/2088] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:42 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:42.421139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ed920-a43f-4d17-9514-5be9a435eeef None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.432310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ed920-a43f-4d17-9514-5be9a435eeef None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.443712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:42.493246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e720fd-a228-4c71-acf7-16b3805ee3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.493989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e720fd-a228-4c71-acf7-16b3805ee3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.494274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e720fd-a228-4c71-acf7-16b3805ee3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.494685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e720fd-a228-4c71-acf7-16b3805ee3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.495789 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e720fd-a228-4c71-acf7-16b3805ee3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.497987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e720fd-a228-4c71-acf7-16b3805ee3d4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.518673 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1032/2089] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:42 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:42.522725 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20480681-15fa-428a-9991-94e6b7c7bb93 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.533141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20480681-15fa-428a-9991-94e6b7c7bb93 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.538158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f444c-6377-4b6a-bd7d-d185d034c7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.538744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f444c-6377-4b6a-bd7d-d185d034c7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.539018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f444c-6377-4b6a-bd7d-d185d034c7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.539285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f444c-6377-4b6a-bd7d-d185d034c7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.540077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f444c-6377-4b6a-bd7d-d185d034c7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.541686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f444c-6377-4b6a-bd7d-d185d034c7b9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.553462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1058/2090] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:42.557326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e720fd-a228-4c71-acf7-16b3805ee3d4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.568073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e720fd-a228-4c71-acf7-16b3805ee3d4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.572770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d5ed8-7d86-43b7-bf7c-d9e69eb09d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.573396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5ed8-7d86-43b7-bf7c-d9e69eb09d46 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.573671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5ed8-7d86-43b7-bf7c-d9e69eb09d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.573941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5ed8-7d86-43b7-bf7c-d9e69eb09d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.574588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d5ed8-7d86-43b7-bf7c-d9e69eb09d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.576174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d5ed8-7d86-43b7-bf7c-d9e69eb09d46 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.586380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1033/2091] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:42.590586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f444c-6377-4b6a-bd7d-d185d034c7b9 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.600959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f444c-6377-4b6a-bd7d-d185d034c7b9 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.605574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669496f-9375-4ed8-8774-5102d480c19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.606247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669496f-9375-4ed8-8774-5102d480c19c admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.606518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669496f-9375-4ed8-8774-5102d480c19c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.606784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669496f-9375-4ed8-8774-5102d480c19c admin admin] PATH_INFO: `/v3/projects/5ab4776e59364a77b26db66c9959b631/users/7f9e1d8ff2c342d2b5f2aae7ce3ff85c/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.623917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669496f-9375-4ed8-8774-5102d480c19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab4776e59364a77b26db66c9959b631, user_id=7f9e1d8ff2c342d2b5f2aae7ce3ff85c, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.626609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669496f-9375-4ed8-8774-5102d480c19c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.638896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1059/2092] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:42 2022] PUT /identity/v3/projects/5ab4776e59364a77b26db66c9959b631/users/7f9e1d8ff2c342d2b5f2aae7ce3ff85c/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:42.649013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55d593-4963-487e-badb-7d966f11b00d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.649328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55d593-4963-487e-badb-7d966f11b00d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.649603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55d593-4963-487e-badb-7d966f11b00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.677957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9c55d593-4963-487e-badb-7d966f11b00d None None] MFA Rules not processed for user `7f9e1d8ff2c342d2b5f2aae7ce3ff85c`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:42.679231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9c55d593-4963-487e-badb-7d966f11b00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:42.744679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1034/2093] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:42.749574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669496f-9375-4ed8-8774-5102d480c19c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.796048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:42.810715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669496f-9375-4ed8-8774-5102d480c19c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.815264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d8298-4c1c-4daf-a946-b496c4194bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.816038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d8298-4c1c-4daf-a946-b496c4194bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.816311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d8298-4c1c-4daf-a946-b496c4194bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.816573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d8298-4c1c-4daf-a946-b496c4194bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.817275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d8298-4c1c-4daf-a946-b496c4194bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.819140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d8298-4c1c-4daf-a946-b496c4194bd8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.848642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1060/2094] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:42 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:42.853023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55d593-4963-487e-badb-7d966f11b00d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.865942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55d593-4963-487e-badb-7d966f11b00d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.941645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c23c2e-1428-4b2c-b605-007d47a06540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.942520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c23c2e-1428-4b2c-b605-007d47a06540 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.943074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c23c2e-1428-4b2c-b605-007d47a06540 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.943348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c23c2e-1428-4b2c-b605-007d47a06540 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.943992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c23c2e-1428-4b2c-b605-007d47a06540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.945973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c23c2e-1428-4b2c-b605-007d47a06540 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:42.971137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1035/2095] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:42 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:42.975499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d8298-4c1c-4daf-a946-b496c4194bd8 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:42.986466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d8298-4c1c-4daf-a946-b496c4194bd8 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:42.995442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c613d-72d1-40b0-af00-3a0454d9a503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:42.996070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c613d-72d1-40b0-af00-3a0454d9a503 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:42.996493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c613d-72d1-40b0-af00-3a0454d9a503 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:42.996827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c613d-72d1-40b0-af00-3a0454d9a503 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:42.997774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c613d-72d1-40b0-af00-3a0454d9a503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:42.999362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c613d-72d1-40b0-af00-3a0454d9a503 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:43.017312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1061/2096] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:43.018940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c23c2e-1428-4b2c-b605-007d47a06540 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:43.029791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c23c2e-1428-4b2c-b605-007d47a06540 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:43.035352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd78bd-c967-4b52-ba18-69129c7a57ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:43.036086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd78bd-c967-4b52-ba18-69129c7a57ca admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:43.036429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd78bd-c967-4b52-ba18-69129c7a57ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:43.036765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd78bd-c967-4b52-ba18-69129c7a57ca admin admin] PATH_INFO: `/v3/projects/52a27104ddb445de87953b6a85a41797/users/01b3240fd64f4e519f6e14f9cab972ca/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:43.050448 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:43.054828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd78bd-c967-4b52-ba18-69129c7a57ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a27104ddb445de87953b6a85a41797, user_id=01b3240fd64f4e519f6e14f9cab972ca, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:43.058050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd78bd-c967-4b52-ba18-69129c7a57ca admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:43.080168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1036/2097] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:43 2022] PUT /identity/v3/projects/52a27104ddb445de87953b6a85a41797/users/01b3240fd64f4e519f6e14f9cab972ca/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:43.084456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c613d-72d1-40b0-af00-3a0454d9a503 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:43.094876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c613d-72d1-40b0-af00-3a0454d9a503 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:43.165492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fcf26-f267-4d26-ae0b-586ef43c5426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:43.166328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fcf26-f267-4d26-ae0b-586ef43c5426 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:43.166611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fcf26-f267-4d26-ae0b-586ef43c5426 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:43.166891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fcf26-f267-4d26-ae0b-586ef43c5426 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:43.167531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fcf26-f267-4d26-ae0b-586ef43c5426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:43.169142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fcf26-f267-4d26-ae0b-586ef43c5426 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:43.179756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1062/2098] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:43.185370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd78bd-c967-4b52-ba18-69129c7a57ca admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:43.196236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd78bd-c967-4b52-ba18-69129c7a57ca admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:43.207554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:43.208301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:43.208589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:43.208871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] PATH_INFO: `/v3/projects/52a27104ddb445de87953b6a85a41797/users/01b3240fd64f4e519f6e14f9cab972ca/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:43.211355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=52a27104ddb445de87953b6a85a41797, user_id=01b3240fd64f4e519f6e14f9cab972ca, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:43.215703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:43.230680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1037/2099] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:43 2022] PUT /identity/v3/projects/52a27104ddb445de87953b6a85a41797/users/01b3240fd64f4e519f6e14f9cab972ca/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:43.241717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27980a67-ec11-4789-b692-3ca5bdef59ed None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:43.242006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27980a67-ec11-4789-b692-3ca5bdef59ed None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:43.242280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27980a67-ec11-4789-b692-3ca5bdef59ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:43.270646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-27980a67-ec11-4789-b692-3ca5bdef59ed None None] MFA Rules not processed for user `01b3240fd64f4e519f6e14f9cab972ca`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:43.271844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-27980a67-ec11-4789-b692-3ca5bdef59ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:43.341354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1063/2100] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:43 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:43.351977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:43.353710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:43.413811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6193e46-2d9b-49c6-bca9-8afacd55e0ba admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:43.417442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:43.418036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:43.418307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:43.418572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:43.427312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:43.428734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:43.438296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1038/2101] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:43.930211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27980a67-ec11-4789-b692-3ca5bdef59ed None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:43.931789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-27980a67-ec11-4789-b692-3ca5bdef59ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:43.944459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27980a67-ec11-4789-b692-3ca5bdef59ed None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:43.948077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a8e5a-938c-4e50-b2ec-d6e0ce2847b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:43.948794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a8e5a-938c-4e50-b2ec-d6e0ce2847b3 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:43.949152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a8e5a-938c-4e50-b2ec-d6e0ce2847b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:43.949429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a8e5a-938c-4e50-b2ec-d6e0ce2847b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:43.990482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:44.008403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a8e5a-938c-4e50-b2ec-d6e0ce2847b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:44.009960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a8e5a-938c-4e50-b2ec-d6e0ce2847b3 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:44.019705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1064/2102] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:45.085966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:45.087384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:45.164488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0defb9-a8e6-4483-a190-d6ce95e9916e service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:45.169522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:45.170160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:45.170424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:45.170696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] PATH_INFO: `/v3/users/a7b7352e0ff74b18a3f9ef9954986973` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:45.171804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b7352e0ff74b18a3f9ef9954986973)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:45.173791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:45.230466 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b7352e0ff74b18a3f9ef9954986973'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.231187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b7352e0ff74b18a3f9ef9954986973'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.238734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b7352e0ff74b18a3f9ef9954986973'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.254257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b7352e0ff74b18a3f9ef9954986973'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.272330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b7352e0ff74b18a3f9ef9954986973'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.275817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:45.283391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1039/2103] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:45 2022] DELETE /identity/v3/users/a7b7352e0ff74b18a3f9ef9954986973 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:45.288466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10a54-eae4-42a5-90cc-7d81cb75b005 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:45.288984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10a54-eae4-42a5-90cc-7d81cb75b005 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:45.289095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f10a54-eae4-42a5-90cc-7d81cb75b005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:45.318587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-57f10a54-eae4-42a5-90cc-7d81cb75b005 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:45.319818 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-57f10a54-eae4-42a5-90cc-7d81cb75b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:45.389850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1065/2104] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:45.401642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:45.403409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:45.465850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb2e89-95e5-4941-8823-93b19bcc7bdb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:45.470132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e05dc4-223c-44e7-a1f1-af83ccae7d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:45.470873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05dc4-223c-44e7-a1f1-af83ccae7d17 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:45.471173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05dc4-223c-44e7-a1f1-af83ccae7d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:45.471443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05dc4-223c-44e7-a1f1-af83ccae7d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:45.493894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e05dc4-223c-44e7-a1f1-af83ccae7d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:45.493894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e05dc4-223c-44e7-a1f1-af83ccae7d17 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:45.508871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1040/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:45.813995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f10a54-eae4-42a5-90cc-7d81cb75b005 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:45.816085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-57f10a54-eae4-42a5-90cc-7d81cb75b005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:45.828936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f10a54-eae4-42a5-90cc-7d81cb75b005 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:45.833873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:45.834513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:45.834793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:45.835079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] PATH_INFO: `/v3/projects/15de188cf537408895579b4fe30ebec1` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:45.836212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15de188cf537408895579b4fe30ebec1)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:45.838189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:45.885686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15de188cf537408895579b4fe30ebec1', 'request_id': 'req-90a6399a-c9b9-4ef4-ac76-7a80669496d4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.891871 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15de188cf537408895579b4fe30ebec1', 'request_id': 'req-90a6399a-c9b9-4ef4-ac76-7a80669496d4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.899853 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15de188cf537408895579b4fe30ebec1', 'request_id': 'req-90a6399a-c9b9-4ef4-ac76-7a80669496d4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.912501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] The token cache is being invalidate because project 15de188cf537408895579b4fe30ebec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:45.912793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.913433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-90a6399a-c9b9-4ef4-ac76-7a80669496d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:45.917091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1066/2106] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:45 2022] DELETE /identity/v3/projects/15de188cf537408895579b4fe30ebec1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:45.919777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facdf58-7b11-404f-ad43-3c40d990df24 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:45.920062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facdf58-7b11-404f-ad43-3c40d990df24 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:45.920333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1facdf58-7b11-404f-ad43-3c40d990df24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:45.940625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1facdf58-7b11-404f-ad43-3c40d990df24 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:45.941835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1facdf58-7b11-404f-ad43-3c40d990df24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:46.003089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1041/2107] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:46.010777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d5286-3079-4b9d-8c85-28045dba2a83 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.011062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d5286-3079-4b9d-8c85-28045dba2a83 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.011334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d5286-3079-4b9d-8c85-28045dba2a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.031231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f14d5286-3079-4b9d-8c85-28045dba2a83 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:46.032346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f14d5286-3079-4b9d-8c85-28045dba2a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:46.040163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1067/2108] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:46.044497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facdf58-7b11-404f-ad43-3c40d990df24 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:46.054410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1facdf58-7b11-404f-ad43-3c40d990df24 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:46.059023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:46.059615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.059902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.060167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.060810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:46.061341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:46.062933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:46.081425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1042/2109] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:46.083564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d5286-3079-4b9d-8c85-28045dba2a83 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:46.098586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d5286-3079-4b9d-8c85-28045dba2a83 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:46.103335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1a959-994f-4207-a751-7794c1a714ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:46.103942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1a959-994f-4207-a751-7794c1a714ae admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.104219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1a959-994f-4207-a751-7794c1a714ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.104489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1a959-994f-4207-a751-7794c1a714ae admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.105148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1a959-994f-4207-a751-7794c1a714ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:46.107071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1a959-994f-4207-a751-7794c1a714ae admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:46.137458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1068/2110] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:46 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:46.141639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:46.151861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55c221-c92e-40ce-a8fe-fbdd84fda3dd admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:46.207070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a81fa-f174-4f9e-ae26-fd5a38fb8b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:46.207665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a81fa-f174-4f9e-ae26-fd5a38fb8b27 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.207941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a81fa-f174-4f9e-ae26-fd5a38fb8b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.208212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a81fa-f174-4f9e-ae26-fd5a38fb8b27 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.208828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a81fa-f174-4f9e-ae26-fd5a38fb8b27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:46.210763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a81fa-f174-4f9e-ae26-fd5a38fb8b27 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:46.231860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1043/2111] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:46 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:46.235914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1a959-994f-4207-a751-7794c1a714ae admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:46.246577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1a959-994f-4207-a751-7794c1a714ae admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:46.251325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9405f98-f2bc-4665-8dd2-8c3ab38b1b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:46.251910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9405f98-f2bc-4665-8dd2-8c3ab38b1b56 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.252184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9405f98-f2bc-4665-8dd2-8c3ab38b1b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.252477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9405f98-f2bc-4665-8dd2-8c3ab38b1b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.253134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9405f98-f2bc-4665-8dd2-8c3ab38b1b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:46.254699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9405f98-f2bc-4665-8dd2-8c3ab38b1b56 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:46.264357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1069/2112] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:46.268350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a81fa-f174-4f9e-ae26-fd5a38fb8b27 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:46.279067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a81fa-f174-4f9e-ae26-fd5a38fb8b27 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:46.284292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23184d-5f49-47d8-b4fd-f2b4b0043820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:46.284824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23184d-5f49-47d8-b4fd-f2b4b0043820 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.285138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23184d-5f49-47d8-b4fd-f2b4b0043820 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.285411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23184d-5f49-47d8-b4fd-f2b4b0043820 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.286045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23184d-5f49-47d8-b4fd-f2b4b0043820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:46.287622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23184d-5f49-47d8-b4fd-f2b4b0043820 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:46.298248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1044/2113] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:46.302957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9405f98-f2bc-4665-8dd2-8c3ab38b1b56 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:46.314280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9405f98-f2bc-4665-8dd2-8c3ab38b1b56 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:46.319063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:46.319709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.319985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.320374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] PATH_INFO: `/v3/projects/f61b5a9aba3d43a9a65f00c475a882cb/users/5bd8f221468f4ca18c0f6c13a6d3b393/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.338031 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61b5a9aba3d43a9a65f00c475a882cb, user_id=5bd8f221468f4ca18c0f6c13a6d3b393, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:46.341207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:46.351340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:46.358262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1070/2114] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:46 2022] PUT /identity/v3/projects/f61b5a9aba3d43a9a65f00c475a882cb/users/5bd8f221468f4ca18c0f6c13a6d3b393/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:46.370972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2600b-d804-4c4a-9644-a72e54cb7893 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.371311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2600b-d804-4c4a-9644-a72e54cb7893 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.371667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2600b-d804-4c4a-9644-a72e54cb7893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.385838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:46.400586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1ec2600b-d804-4c4a-9644-a72e54cb7893 None None] MFA Rules not processed for user `5bd8f221468f4ca18c0f6c13a6d3b393`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:46.401893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1ec2600b-d804-4c4a-9644-a72e54cb7893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:46.466862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1045/2115] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:46.480473 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:46.482214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:46.545373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50751e40-59e7-4b76-9195-0d869feea4fd admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:46.549004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029447e782484e8f8a704b683ede1819', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:46.549676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:46.549957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:46.550231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:46.558746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:46.560172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:46.569616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1071/2116] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:48.747510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2600b-d804-4c4a-9644-a72e54cb7893 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:48.750269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1ec2600b-d804-4c4a-9644-a72e54cb7893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:48.910599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2600b-d804-4c4a-9644-a72e54cb7893 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:48.926698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:48.927494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:48.927879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:48.928376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] PATH_INFO: `/v3/users/522d7bd22f2f48b9bd0c8d02945ebe04` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:48.931297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=522d7bd22f2f48b9bd0c8d02945ebe04)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:48.935184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:48.941256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:48.943361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:48.971254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f731-7067-4eaa-8092-fe9218d24d51 service glance] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:48.977776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:48.978791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:48.979270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:48.979828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] PATH_INFO: `/v3/users/2d31629a654247f2916cd09ec3f3681e` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:48.981349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d31629a654247f2916cd09ec3f3681e)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:48.984223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:49.015402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522d7bd22f2f48b9bd0c8d02945ebe04'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.016237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522d7bd22f2f48b9bd0c8d02945ebe04'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.028870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522d7bd22f2f48b9bd0c8d02945ebe04'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.044945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522d7bd22f2f48b9bd0c8d02945ebe04'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.054191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-08970c1d-5875-4162-bf49-c0800daf5529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522d7bd22f2f48b9bd0c8d02945ebe04'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.063855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1046/2117] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:48 2022] DELETE /identity/v3/users/522d7bd22f2f48b9bd0c8d02945ebe04 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:49.064169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d31629a654247f2916cd09ec3f3681e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.065487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d31629a654247f2916cd09ec3f3681e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.066453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0993e-3e2d-4eff-8b0a-881922db7e68 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:49.066872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0993e-3e2d-4eff-8b0a-881922db7e68 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:49.067274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0993e-3e2d-4eff-8b0a-881922db7e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:49.084034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d31629a654247f2916cd09ec3f3681e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.088063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-72e0993e-3e2d-4eff-8b0a-881922db7e68 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:49.089413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-72e0993e-3e2d-4eff-8b0a-881922db7e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:49.093894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d31629a654247f2916cd09ec3f3681e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.094692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66557698-4c4c-46cc-8fe6-1e4970f29524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d31629a654247f2916cd09ec3f3681e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:49.104413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:49.114052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1072/2118] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:48 2022] DELETE /identity/v3/users/2d31629a654247f2916cd09ec3f3681e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:49.115440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15e6be-9091-4ac8-a1e0-f53790ab42de None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:49.115727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15e6be-9091-4ac8-a1e0-f53790ab42de None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:49.116003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15e6be-9091-4ac8-a1e0-f53790ab42de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:49.141405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ca15e6be-9091-4ac8-a1e0-f53790ab42de None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:49.142939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ca15e6be-9091-4ac8-a1e0-f53790ab42de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:49.169098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1047/2119] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:49.181660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0993e-3e2d-4eff-8b0a-881922db7e68 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:49.183864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-72e0993e-3e2d-4eff-8b0a-881922db7e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:49.219069 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1073/2120] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:49.232343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15e6be-9091-4ac8-a1e0-f53790ab42de None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:49.233594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ca15e6be-9091-4ac8-a1e0-f53790ab42de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:49.256626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:49.260901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0993e-3e2d-4eff-8b0a-881922db7e68 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:49.342936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15e6be-9091-4ac8-a1e0-f53790ab42de None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:49.347433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:49.348839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:49.349179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:49.349646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:49.350009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:49.353730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:49.354265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:49.354548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:49.381963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:49.383566 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:49.395204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1048/2121] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:49.402833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:49.404260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:49.416379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1074/2122] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:50.074599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:50.076707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:50.093442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f71e62-4bd8-4cbf-9e69-f562dd015112 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:50.101049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:50.101975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:50.102390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:50.103020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] PATH_INFO: `/v3/projects/dec55f50470542ffbfd84b94acd17419` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:50.104416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec55f50470542ffbfd84b94acd17419)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:50.106541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:50.157961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:50.160044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:50.180464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e81783-7c1d-4334-9734-fedc3830c994 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:50.182771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec55f50470542ffbfd84b94acd17419', 'request_id': 'req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.193971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec55f50470542ffbfd84b94acd17419', 'request_id': 'req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.206028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec55f50470542ffbfd84b94acd17419', 'request_id': 'req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.215543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:50.222667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] The token cache is being invalidate because project dec55f50470542ffbfd84b94acd17419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:50.222968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.224735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.226418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1049/2123] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:50 2022] DELETE /identity/v3/projects/dec55f50470542ffbfd84b94acd17419 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:50.229201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:50.229201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:50.247020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:50.248303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:50.248747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:50.249084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] PATH_INFO: `/v3/projects/ee6904f697d34b7da45fe23f883d2084` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:50.251383 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee6904f697d34b7da45fe23f883d2084)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:50.253805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:50.303963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b0d75-30b5-4ed3-ba0a-2e8e9080a3ef admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:50.318545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee6904f697d34b7da45fe23f883d2084', 'request_id': 'req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.325178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee6904f697d34b7da45fe23f883d2084', 'request_id': 'req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.331212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee6904f697d34b7da45fe23f883d2084', 'request_id': 'req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.343030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] The token cache is being invalidate because project ee6904f697d34b7da45fe23f883d2084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:50.343317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.344056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.345622 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1075/2124] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:50 2022] DELETE /identity/v3/projects/ee6904f697d34b7da45fe23f883d2084 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:50.347101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:50.348367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:50.362518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:50.363298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:50.363577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:50.363845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:50.365640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:50.403611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:50.436276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:50.436858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6d3a7-7f51-4f5e-8a77-c16c3ee2e3d1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:50.438117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:50.442418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:50.443912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:50.444203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:50.444481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] PATH_INFO: `/v3/users/bb0b6c7db1b54542b7e37c24bed9d09b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:50.445683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0b6c7db1b54542b7e37c24bed9d09b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:50.447644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:50.448223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1050/2125] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:50.449143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:50.450442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:50.465937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6f490-1d53-4e93-95c1-ca2aa526669b service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:50.472483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:50.473268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:50.473548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:50.473821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] PATH_INFO: `/v3/users/41c05e9355f244d0b421cfdd170988cf` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:50.475091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c05e9355f244d0b421cfdd170988cf)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:50.478005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:50.523216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0b6c7db1b54542b7e37c24bed9d09b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.524116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0b6c7db1b54542b7e37c24bed9d09b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.537647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c05e9355f244d0b421cfdd170988cf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.538753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c05e9355f244d0b421cfdd170988cf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.541671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0b6c7db1b54542b7e37c24bed9d09b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.547480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c05e9355f244d0b421cfdd170988cf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.550768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0b6c7db1b54542b7e37c24bed9d09b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.551429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0b6c7db1b54542b7e37c24bed9d09b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.556712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:50.562986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c05e9355f244d0b421cfdd170988cf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.569120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1076/2126] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:50 2022] DELETE /identity/v3/users/bb0b6c7db1b54542b7e37c24bed9d09b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:50.573115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c05e9355f244d0b421cfdd170988cf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:50.579907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1051/2127] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:50 2022] DELETE /identity/v3/users/41c05e9355f244d0b421cfdd170988cf => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:50.897307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:50.900610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:50.901042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:50.905367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.009402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d22c-276f-4f74-b87e-3eb5eebda3fd admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.015934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.017627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.017627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.017627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] PATH_INFO: `/v3/projects/e5fd1d7a457449859f4b9f7c3b297b79` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.019574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5fd1d7a457449859f4b9f7c3b297b79)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.020172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebb556-6603-418b-a6f9-eeae9dcb9a04 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.023054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.027853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.028546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.028839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.029186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] PATH_INFO: `/v3/projects/24d0607c961b47a2925ab6dd57a50143` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.030468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d0607c961b47a2925ab6dd57a50143)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.032576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.103022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5fd1d7a457449859f4b9f7c3b297b79', 'request_id': 'req-beb248fa-b892-4f3d-8bf3-39d444ff9a96'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.108913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d0607c961b47a2925ab6dd57a50143', 'request_id': 'req-1ba01e22-18ab-4105-a91d-7a16be45afa0'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.112010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5fd1d7a457449859f4b9f7c3b297b79', 'request_id': 'req-beb248fa-b892-4f3d-8bf3-39d444ff9a96'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.118600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5fd1d7a457449859f4b9f7c3b297b79', 'request_id': 'req-beb248fa-b892-4f3d-8bf3-39d444ff9a96'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.118881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d0607c961b47a2925ab6dd57a50143', 'request_id': 'req-1ba01e22-18ab-4105-a91d-7a16be45afa0'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.128638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:51.130626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d0607c961b47a2925ab6dd57a50143', 'request_id': 'req-1ba01e22-18ab-4105-a91d-7a16be45afa0'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.135819 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] The token cache is being invalidate because project e5fd1d7a457449859f4b9f7c3b297b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:51.136372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.138054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-beb248fa-b892-4f3d-8bf3-39d444ff9a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.149280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1077/2128] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:50 2022] DELETE /identity/v3/projects/e5fd1d7a457449859f4b9f7c3b297b79 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:51.149280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7994a1-7f45-49f0-9e40-d6b943bd3222 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.149280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7994a1-7f45-49f0-9e40-d6b943bd3222 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.149280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7994a1-7f45-49f0-9e40-d6b943bd3222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.160743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] The token cache is being invalidate because project 24d0607c961b47a2925ab6dd57a50143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:51.161413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.162659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.166205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1052/2129] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:50 2022] DELETE /identity/v3/projects/24d0607c961b47a2925ab6dd57a50143 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:51.170808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ea7994a1-7f45-49f0-9e40-d6b943bd3222 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:51.172072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.172316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ea7994a1-7f45-49f0-9e40-d6b943bd3222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.174158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.262632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba01e22-18ab-4105-a91d-7a16be45afa0 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.269146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.269859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.270215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.270553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] PATH_INFO: `/v3/users/8ec618778cad4388a038b24451c774c9` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.271807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec618778cad4388a038b24451c774c9)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.274442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.275700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1078/2130] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:51.287467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c9127-bbab-44ee-a396-8df49d11e036 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.287746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c9127-bbab-44ee-a396-8df49d11e036 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.288021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c9127-bbab-44ee-a396-8df49d11e036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.294087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:51.310134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1f2c9127-bbab-44ee-a396-8df49d11e036 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:51.311352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1f2c9127-bbab-44ee-a396-8df49d11e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.324564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1079/2131] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:51.328712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c9127-bbab-44ee-a396-8df49d11e036 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.339776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c9127-bbab-44ee-a396-8df49d11e036 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.345672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.346374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec618778cad4388a038b24451c774c9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.347338 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec618778cad4388a038b24451c774c9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.347451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.347451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.347874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.348313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:51.348642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.350231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.357014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec618778cad4388a038b24451c774c9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.369667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1080/2132] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:51.373965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec618778cad4388a038b24451c774c9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.374484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.382571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec618778cad4388a038b24451c774c9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.387512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0143c-7b79-4a2c-8fd0-0332b81380f4 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.390262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1053/2133] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:51 2022] DELETE /identity/v3/users/8ec618778cad4388a038b24451c774c9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:51.397627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719654fc-1856-488d-8c99-b8c1999aef94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.398442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719654fc-1856-488d-8c99-b8c1999aef94 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.398718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719654fc-1856-488d-8c99-b8c1999aef94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.398984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719654fc-1856-488d-8c99-b8c1999aef94 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.399770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719654fc-1856-488d-8c99-b8c1999aef94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.402361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719654fc-1856-488d-8c99-b8c1999aef94 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.449704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1081/2134] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:51.454357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.458711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.553236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0231f-9943-4dd1-89fa-b935e8727e9f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.625034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9baae-0ec4-4ae8-ada1-57da0926636b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.625683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9baae-0ec4-4ae8-ada1-57da0926636b admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.625965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9baae-0ec4-4ae8-ada1-57da0926636b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.626239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9baae-0ec4-4ae8-ada1-57da0926636b admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.627162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9baae-0ec4-4ae8-ada1-57da0926636b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.629104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9baae-0ec4-4ae8-ada1-57da0926636b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.650905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1054/2135] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:51.655017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719654fc-1856-488d-8c99-b8c1999aef94 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.666235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719654fc-1856-488d-8c99-b8c1999aef94 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.670830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.671431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.671727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.671993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.672627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.675575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.687459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1082/2136] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:51.691481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9baae-0ec4-4ae8-ada1-57da0926636b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.701438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.702662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9baae-0ec4-4ae8-ada1-57da0926636b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.704076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.711635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49822701-ae8f-49e4-990b-4c482d1597f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.712347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49822701-ae8f-49e4-990b-4c482d1597f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.712642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49822701-ae8f-49e4-990b-4c482d1597f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.713183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49822701-ae8f-49e4-990b-4c482d1597f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.714114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49822701-ae8f-49e4-990b-4c482d1597f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.716551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49822701-ae8f-49e4-990b-4c482d1597f3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.729615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b138c-44ba-49e7-a165-548e324d3696 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.729873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1055/2137] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:51.734838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.735007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49822701-ae8f-49e4-990b-4c482d1597f3 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.735487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.735768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.736110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] PATH_INFO: `/v3/projects/c2f98ab4a21c4e44a58ba2b93115535a` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.737334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f98ab4a21c4e44a58ba2b93115535a)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.739387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.752871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49822701-ae8f-49e4-990b-4c482d1597f3 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.757891 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21c5a3-cc23-4902-94ce-400432f2569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.758554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c5a3-cc23-4902-94ce-400432f2569b admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.758825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c5a3-cc23-4902-94ce-400432f2569b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.759151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c5a3-cc23-4902-94ce-400432f2569b admin admin] PATH_INFO: `/v3/projects/10d31e5821af4430a93689e6505b9b11/users/385fbed74cd74f57bb4a760673b44e47/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.772804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:51.777385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21c5a3-cc23-4902-94ce-400432f2569b admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d31e5821af4430a93689e6505b9b11, user_id=385fbed74cd74f57bb4a760673b44e47, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.781193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f21c5a3-cc23-4902-94ce-400432f2569b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.796539 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1056/2138] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:51 2022] PUT /identity/v3/projects/10d31e5821af4430a93689e6505b9b11/users/385fbed74cd74f57bb4a760673b44e47/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:51.797815 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f98ab4a21c4e44a58ba2b93115535a', 'request_id': 'req-abb07746-ae2b-4cc3-807d-35bd019172f0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.805012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f98ab4a21c4e44a58ba2b93115535a', 'request_id': 'req-abb07746-ae2b-4cc3-807d-35bd019172f0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.808619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abc1b5-147d-4cd4-9975-bc1b9bbe9fc4 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.809030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abc1b5-147d-4cd4-9975-bc1b9bbe9fc4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.809364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abc1b5-147d-4cd4-9975-bc1b9bbe9fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.810907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f98ab4a21c4e44a58ba2b93115535a', 'request_id': 'req-abb07746-ae2b-4cc3-807d-35bd019172f0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.824231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] The token cache is being invalidate because project c2f98ab4a21c4e44a58ba2b93115535a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:51.824519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.825195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-abb07746-ae2b-4cc3-807d-35bd019172f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:51.826724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1083/2139] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:51 2022] DELETE /identity/v3/projects/c2f98ab4a21c4e44a58ba2b93115535a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:51.832433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239402-4902-4322-8c3e-65cc20af7064 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.832716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239402-4902-4322-8c3e-65cc20af7064 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.832992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44239402-4902-4322-8c3e-65cc20af7064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.838445 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-00abc1b5-147d-4cd4-9975-bc1b9bbe9fc4 None None] MFA Rules not processed for user `385fbed74cd74f57bb4a760673b44e47`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:51.839677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-00abc1b5-147d-4cd4-9975-bc1b9bbe9fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.854394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-44239402-4902-4322-8c3e-65cc20af7064 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:51.855659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-44239402-4902-4322-8c3e-65cc20af7064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.883248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:51.912243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1057/2140] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:51.916751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abc1b5-147d-4cd4-9975-bc1b9bbe9fc4 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.918110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-00abc1b5-147d-4cd4-9975-bc1b9bbe9fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.927494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1084/2141] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:51.931343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abc1b5-147d-4cd4-9975-bc1b9bbe9fc4 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:51.935020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28325-dd7d-4833-a461-348cb7eabe12 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.935302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28325-dd7d-4833-a461-348cb7eabe12 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.935572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf28325-dd7d-4833-a461-348cb7eabe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.936445 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e8e00-041e-4ba2-8233-ca7b9d52e6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:51.937261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e8e00-041e-4ba2-8233-ca7b9d52e6cb admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:51.937548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e8e00-041e-4ba2-8233-ca7b9d52e6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:51.937843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e8e00-041e-4ba2-8233-ca7b9d52e6cb admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:51.938470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e8e00-041e-4ba2-8233-ca7b9d52e6cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:51.940535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e8e00-041e-4ba2-8233-ca7b9d52e6cb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:51.954665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1cf28325-dd7d-4833-a461-348cb7eabe12 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:51.956423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1cf28325-dd7d-4833-a461-348cb7eabe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:51.964729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1085/2142] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:51.968715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf28325-dd7d-4833-a461-348cb7eabe12 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.972471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1058/2143] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:51.973331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e8e00-041e-4ba2-8233-ca7b9d52e6cb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:51.978886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf28325-dd7d-4833-a461-348cb7eabe12 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.039782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e8e00-041e-4ba2-8233-ca7b9d52e6cb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.044388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d797d44-a97a-49d2-a840-b53a9fab8463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.044997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d797d44-a97a-49d2-a840-b53a9fab8463 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.045311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d797d44-a97a-49d2-a840-b53a9fab8463 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.045461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.045607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d797d44-a97a-49d2-a840-b53a9fab8463 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.046070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.046211 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d797d44-a97a-49d2-a840-b53a9fab8463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.046346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.046609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.047246 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:52.047546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.048101 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d797d44-a97a-49d2-a840-b53a9fab8463 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.049224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.060634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1086/2144] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:49:52.065187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.070585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1059/2145] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:51 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:52.074384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d797d44-a97a-49d2-a840-b53a9fab8463 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.075776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9e39d-bb02-44c1-b052-370dfedabfc6 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.081703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b7b49-b3e3-4297-9215-90140c3e8434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.082330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b7b49-b3e3-4297-9215-90140c3e8434 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.082730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b7b49-b3e3-4297-9215-90140c3e8434 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.083066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b7b49-b3e3-4297-9215-90140c3e8434 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.083643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b7b49-b3e3-4297-9215-90140c3e8434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.084761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d797d44-a97a-49d2-a840-b53a9fab8463 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.085729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b7b49-b3e3-4297-9215-90140c3e8434 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.089970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f23b7-d8e0-451f-8e62-1ac05dcc4407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.091775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f23b7-d8e0-451f-8e62-1ac05dcc4407 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.092067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f23b7-d8e0-451f-8e62-1ac05dcc4407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.092341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f23b7-d8e0-451f-8e62-1ac05dcc4407 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.093028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f23b7-d8e0-451f-8e62-1ac05dcc4407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.094649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f23b7-d8e0-451f-8e62-1ac05dcc4407 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.108676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1060/2146] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:52.112779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f23b7-d8e0-451f-8e62-1ac05dcc4407 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.118709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1087/2147] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:52 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:52.119525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b7b49-b3e3-4297-9215-90140c3e8434 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.123896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f23b7-d8e0-451f-8e62-1ac05dcc4407 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.163873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:52.189231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38c10-3ebe-4d56-9e08-2ead97153ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.190534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38c10-3ebe-4d56-9e08-2ead97153ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.190814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38c10-3ebe-4d56-9e08-2ead97153ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.191083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38c10-3ebe-4d56-9e08-2ead97153ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.191776 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce38c10-3ebe-4d56-9e08-2ead97153ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.193430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce38c10-3ebe-4d56-9e08-2ead97153ef3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.193573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b7b49-b3e3-4297-9215-90140c3e8434 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.198912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9745866-47e0-4189-ab1f-69c57b54771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.199517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9745866-47e0-4189-ab1f-69c57b54771c admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.199804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9745866-47e0-4189-ab1f-69c57b54771c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.200077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9745866-47e0-4189-ab1f-69c57b54771c admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.200737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9745866-47e0-4189-ab1f-69c57b54771c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.202679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9745866-47e0-4189-ab1f-69c57b54771c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.203494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1061/2148] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:52.208302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38c10-3ebe-4d56-9e08-2ead97153ef3 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.218195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38c10-3ebe-4d56-9e08-2ead97153ef3 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.222859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074f90-9e23-4b69-b6af-6a95820985d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.223495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074f90-9e23-4b69-b6af-6a95820985d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.223775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074f90-9e23-4b69-b6af-6a95820985d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.224047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e074f90-9e23-4b69-b6af-6a95820985d8 admin admin] PATH_INFO: `/v3/projects/fcaf7d9a0aac4cfda8badbdd9848e12e/users/3dff49e6f1db4fd58b49e5aabdccf38d/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.227250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1088/2149] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:52 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:52.228210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9745866-47e0-4189-ab1f-69c57b54771c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.237844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9745866-47e0-4189-ab1f-69c57b54771c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.239002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e074f90-9e23-4b69-b6af-6a95820985d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcaf7d9a0aac4cfda8badbdd9848e12e, user_id=3dff49e6f1db4fd58b49e5aabdccf38d, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.241670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e074f90-9e23-4b69-b6af-6a95820985d8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.242525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873bfd8-0539-4676-abc0-b8fc399dc243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.243106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873bfd8-0539-4676-abc0-b8fc399dc243 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.243384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873bfd8-0539-4676-abc0-b8fc399dc243 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.243656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873bfd8-0539-4676-abc0-b8fc399dc243 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.244284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873bfd8-0539-4676-abc0-b8fc399dc243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.245903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873bfd8-0539-4676-abc0-b8fc399dc243 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.255741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1062/2150] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:52 2022] PUT /identity/v3/projects/fcaf7d9a0aac4cfda8badbdd9848e12e/users/3dff49e6f1db4fd58b49e5aabdccf38d/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:52.260313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1089/2151] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:52.264446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074f90-9e23-4b69-b6af-6a95820985d8 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.265764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9032904-b322-4b42-83d6-d6829e160804 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.266047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9032904-b322-4b42-83d6-d6829e160804 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.266314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9032904-b322-4b42-83d6-d6829e160804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.293229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:52.296876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d9032904-b322-4b42-83d6-d6829e160804 None None] MFA Rules not processed for user `3dff49e6f1db4fd58b49e5aabdccf38d`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:52.298693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d9032904-b322-4b42-83d6-d6829e160804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:52.338400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e074f90-9e23-4b69-b6af-6a95820985d8 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.344039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7da6-9c3f-4de3-a8c7-067f56713017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.344827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7da6-9c3f-4de3-a8c7-067f56713017 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.345206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7da6-9c3f-4de3-a8c7-067f56713017 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.345522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b7da6-9c3f-4de3-a8c7-067f56713017 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.346228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b7da6-9c3f-4de3-a8c7-067f56713017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.347975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b7da6-9c3f-4de3-a8c7-067f56713017 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.371863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1063/2152] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:52.376321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7da6-9c3f-4de3-a8c7-067f56713017 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.385816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1090/2153] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:49:52.389689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9032904-b322-4b42-83d6-d6829e160804 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.390565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b7da6-9c3f-4de3-a8c7-067f56713017 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.396397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e89afc-5ccd-47af-99f0-0e57809edcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.396705 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e89afc-5ccd-47af-99f0-0e57809edcc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.397048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e89afc-5ccd-47af-99f0-0e57809edcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.397406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e89afc-5ccd-47af-99f0-0e57809edcc5 admin admin] PATH_INFO: `/v3/projects/f4e0c857a31147f8afe25e68cc3e9f24/users/6f366d6ae3b4404f9fa4343913f67998/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.402667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9032904-b322-4b42-83d6-d6829e160804 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.409127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13220a5a-1386-4259-8e0e-1a5ab5068a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.409956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220a5a-1386-4259-8e0e-1a5ab5068a1d admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.410349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220a5a-1386-4259-8e0e-1a5ab5068a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.410834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13220a5a-1386-4259-8e0e-1a5ab5068a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.411409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13220a5a-1386-4259-8e0e-1a5ab5068a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.413404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13220a5a-1386-4259-8e0e-1a5ab5068a1d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.415526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e89afc-5ccd-47af-99f0-0e57809edcc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e0c857a31147f8afe25e68cc3e9f24, user_id=6f366d6ae3b4404f9fa4343913f67998, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.419924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e89afc-5ccd-47af-99f0-0e57809edcc5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.436460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1064/2154] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:52 2022] PUT /identity/v3/projects/f4e0c857a31147f8afe25e68cc3e9f24/users/6f366d6ae3b4404f9fa4343913f67998/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:52.442184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf18ad-2f36-4456-bcdd-0aca9116f15d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.442492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf18ad-2f36-4456-bcdd-0aca9116f15d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.442770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf18ad-2f36-4456-bcdd-0aca9116f15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.443449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1091/2155] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:52 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:52.447599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13220a5a-1386-4259-8e0e-1a5ab5068a1d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.463698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-51cf18ad-2f36-4456-bcdd-0aca9116f15d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:52.464940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-51cf18ad-2f36-4456-bcdd-0aca9116f15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:52.525542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13220a5a-1386-4259-8e0e-1a5ab5068a1d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.552507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685fc22-2d52-4678-8585-d0826db4d206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.552507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685fc22-2d52-4678-8585-d0826db4d206 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.552507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685fc22-2d52-4678-8585-d0826db4d206 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.552507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685fc22-2d52-4678-8585-d0826db4d206 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.553365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685fc22-2d52-4678-8585-d0826db4d206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.555514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685fc22-2d52-4678-8585-d0826db4d206 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.563629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1065/2156] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:52.572910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf18ad-2f36-4456-bcdd-0aca9116f15d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.575871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-51cf18ad-2f36-4456-bcdd-0aca9116f15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:52.602326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1092/2157] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:52 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:52.606165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685fc22-2d52-4678-8585-d0826db4d206 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.617370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:52.627134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685fc22-2d52-4678-8585-d0826db4d206 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.632949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b39456-16f5-4ac1-9107-95235e18c8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.633583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b39456-16f5-4ac1-9107-95235e18c8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.633866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b39456-16f5-4ac1-9107-95235e18c8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.634157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b39456-16f5-4ac1-9107-95235e18c8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.634771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b39456-16f5-4ac1-9107-95235e18c8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.636377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b39456-16f5-4ac1-9107-95235e18c8b3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.649107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1093/2158] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:52.653368 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b39456-16f5-4ac1-9107-95235e18c8b3 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.658712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf18ad-2f36-4456-bcdd-0aca9116f15d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.663012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbefd0c-7afe-441d-add9-acd64f75d730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.663616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefd0c-7afe-441d-add9-acd64f75d730 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.663890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefd0c-7afe-441d-add9-acd64f75d730 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.664162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbefd0c-7afe-441d-add9-acd64f75d730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.667200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b39456-16f5-4ac1-9107-95235e18c8b3 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.675044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbefd0c-7afe-441d-add9-acd64f75d730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.675303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1683a4-6298-4d53-b691-e04f88f90627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.675965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1683a4-6298-4d53-b691-e04f88f90627 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.676444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1683a4-6298-4d53-b691-e04f88f90627 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.676622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbefd0c-7afe-441d-add9-acd64f75d730 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.676744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1683a4-6298-4d53-b691-e04f88f90627 admin admin] PATH_INFO: `/v3/projects/04f9ac1918c040f19061b7a36a46878d/users/a7c8cdfc7ec74b10972bdc28d455dba3/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.694747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1066/2159] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:52.695061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1683a4-6298-4d53-b691-e04f88f90627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f9ac1918c040f19061b7a36a46878d, user_id=a7c8cdfc7ec74b10972bdc28d455dba3, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.697761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1683a4-6298-4d53-b691-e04f88f90627 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.712115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1094/2160] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:52 2022] PUT /identity/v3/projects/04f9ac1918c040f19061b7a36a46878d/users/a7c8cdfc7ec74b10972bdc28d455dba3/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:52.716173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbefd0c-7afe-441d-add9-acd64f75d730 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.728798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbefd0c-7afe-441d-add9-acd64f75d730 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.793544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f94c78-cbd0-429f-baf1-655c1efcadeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.794240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f94c78-cbd0-429f-baf1-655c1efcadeb admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.794585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f94c78-cbd0-429f-baf1-655c1efcadeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.794920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f94c78-cbd0-429f-baf1-655c1efcadeb admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.795622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f94c78-cbd0-429f-baf1-655c1efcadeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.797481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f94c78-cbd0-429f-baf1-655c1efcadeb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.810267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1067/2161] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:52.814192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1683a4-6298-4d53-b691-e04f88f90627 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:52.826732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1683a4-6298-4d53-b691-e04f88f90627 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:52.831521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:52.832286 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.832576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.832877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] PATH_INFO: `/v3/projects/04f9ac1918c040f19061b7a36a46878d/users/a7c8cdfc7ec74b10972bdc28d455dba3/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.835241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f9ac1918c040f19061b7a36a46878d, user_id=a7c8cdfc7ec74b10972bdc28d455dba3, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:52.841096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:52.852193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1095/2162] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:52 2022] PUT /identity/v3/projects/04f9ac1918c040f19061b7a36a46878d/users/a7c8cdfc7ec74b10972bdc28d455dba3/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:52.864789 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e77ea0-002e-4ff0-9276-c5147ca95216 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:52.865090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e77ea0-002e-4ff0-9276-c5147ca95216 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:52.865379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e77ea0-002e-4ff0-9276-c5147ca95216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:52.902591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-18e77ea0-002e-4ff0-9276-c5147ca95216 None None] MFA Rules not processed for user `a7c8cdfc7ec74b10972bdc28d455dba3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:52.903924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-18e77ea0-002e-4ff0-9276-c5147ca95216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:52.994668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:53.000154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1068/2163] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:53.000418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:53.010540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e77ea0-002e-4ff0-9276-c5147ca95216 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:53.012827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-18e77ea0-002e-4ff0-9276-c5147ca95216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:53.064243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:53.093414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:53.101837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e77ea0-002e-4ff0-9276-c5147ca95216 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:53.105733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb26be-e957-4b8c-833b-8683e438401b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:53.106281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb26be-e957-4b8c-833b-8683e438401b service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:53.106679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb26be-e957-4b8c-833b-8683e438401b service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:53.107007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb26be-e957-4b8c-833b-8683e438401b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:53.118467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb26be-e957-4b8c-833b-8683e438401b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:53.120305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb26be-e957-4b8c-833b-8683e438401b service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:53.123417 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2ee08-cd14-4031-bea6-7d51ca5bb722 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:53.128753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:53.129728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:53.130155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:53.130599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] PATH_INFO: `/v3/users/ef784188fe684305817ed12f41bd6cdd` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:53.132162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef784188fe684305817ed12f41bd6cdd)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:53.134434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:53.135770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1069/2164] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:53.203556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb26be-e957-4b8c-833b-8683e438401b service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:53.215825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef784188fe684305817ed12f41bd6cdd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:53.217179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef784188fe684305817ed12f41bd6cdd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:53.223136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb26be-e957-4b8c-833b-8683e438401b service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:53.231601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef784188fe684305817ed12f41bd6cdd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:53.241840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef784188fe684305817ed12f41bd6cdd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:53.243119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:53.243555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-58a2fda4-ad2a-4d92-9494-797c2aab0c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef784188fe684305817ed12f41bd6cdd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:53.244362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:53.244689 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:53.246830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:53.251629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:53.279082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1096/2165] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:52 2022] DELETE /identity/v3/users/ef784188fe684305817ed12f41bd6cdd => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:53.286345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c785-ab2f-45ec-acc2-921b616c8d39 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:53.286569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c785-ab2f-45ec-acc2-921b616c8d39 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:53.286788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c785-ab2f-45ec-acc2-921b616c8d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:53.309731 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6792c785-ab2f-45ec-acc2-921b616c8d39 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:53.311301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6792c785-ab2f-45ec-acc2-921b616c8d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:53.353319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:53.355384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:53.365365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1070/2166] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:49:53.397195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1097/2167] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:53.407419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:53.409708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:53.503088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0665-2d9d-4565-9d0d-7730aac32479 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:53.503628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0665-2d9d-4565-9d0d-7730aac32479 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:53.504056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d0665-2d9d-4565-9d0d-7730aac32479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:53.509042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae225c-b1d9-4126-80ee-202ae5174f0a service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:53.553916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-642d0665-2d9d-4565-9d0d-7730aac32479 None None] MFA Rules not processed for user `6f366d6ae3b4404f9fa4343913f67998`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:53.555888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-642d0665-2d9d-4565-9d0d-7730aac32479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:53.585173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:53.589443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:53.589752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:53.590034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:53.626669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:53.627654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:53.629635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:53.657109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1071/2168] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:53.707686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1098/2169] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:53.767248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:53.769276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:53.863700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:53.869492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b970a-966f-45c5-8551-aeecc2f1e081 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:53.875340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d06c0-0095-468e-8180-f9a86b4cd6eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:53.876259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d06c0-0095-468e-8180-f9a86b4cd6eb service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:53.877509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d06c0-0095-468e-8180-f9a86b4cd6eb service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:53.877811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d06c0-0095-468e-8180-f9a86b4cd6eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:53.890683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d06c0-0095-468e-8180-f9a86b4cd6eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:53.892733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d06c0-0095-468e-8180-f9a86b4cd6eb service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:53.909050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1072/2170] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:54.107167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d0665-2d9d-4565-9d0d-7730aac32479 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:54.108839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-642d0665-2d9d-4565-9d0d-7730aac32479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:54.129642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d0665-2d9d-4565-9d0d-7730aac32479 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:54.137894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:54.138540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:54.138816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:54.139089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] PATH_INFO: `/v3/projects/930d423c40374d72ad0b4fcfcb22221c` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:54.140224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=930d423c40374d72ad0b4fcfcb22221c)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:54.142219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:54.192079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '930d423c40374d72ad0b4fcfcb22221c', 'request_id': 'req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.198319 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '930d423c40374d72ad0b4fcfcb22221c', 'request_id': 'req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.204835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '930d423c40374d72ad0b4fcfcb22221c', 'request_id': 'req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.217571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] The token cache is being invalidate because project 930d423c40374d72ad0b4fcfcb22221c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:54.218170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.218431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.220032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1099/2171] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:54 2022] DELETE /identity/v3/projects/930d423c40374d72ad0b4fcfcb22221c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:54.224552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d06c0-0095-468e-8180-f9a86b4cd6eb service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:54.235716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d06c0-0095-468e-8180-f9a86b4cd6eb service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:54.304077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:54.305030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:54.305613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:54.306081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] PATH_INFO: `/v3/users/7f9e1d8ff2c342d2b5f2aae7ce3ff85c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:54.307832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f9e1d8ff2c342d2b5f2aae7ce3ff85c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:54.310675 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:54.376416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f9e1d8ff2c342d2b5f2aae7ce3ff85c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.377430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f9e1d8ff2c342d2b5f2aae7ce3ff85c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.388341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f9e1d8ff2c342d2b5f2aae7ce3ff85c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.400754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f9e1d8ff2c342d2b5f2aae7ce3ff85c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.408158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f9e1d8ff2c342d2b5f2aae7ce3ff85c'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.414846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1073/2172] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:54 2022] DELETE /identity/v3/users/7f9e1d8ff2c342d2b5f2aae7ce3ff85c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:54.707204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:54.708836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:54.722303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6bb66-9c7d-4d27-8fa2-a183d26eabf3 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:54.790332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:54.790967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:54.791243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:54.791512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] PATH_INFO: `/v3/projects/5ab4776e59364a77b26db66c9959b631` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:54.792628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab4776e59364a77b26db66c9959b631)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:54.794650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:54.842412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab4776e59364a77b26db66c9959b631', 'request_id': 'req-626f3039-a62b-4a64-8861-32271f14f8e4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.848792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab4776e59364a77b26db66c9959b631', 'request_id': 'req-626f3039-a62b-4a64-8861-32271f14f8e4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.850833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:54.859144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab4776e59364a77b26db66c9959b631', 'request_id': 'req-626f3039-a62b-4a64-8861-32271f14f8e4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.873151 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] The token cache is being invalidate because project 5ab4776e59364a77b26db66c9959b631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:54.873456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.874116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:54.875672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1100/2173] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:54 2022] DELETE /identity/v3/projects/5ab4776e59364a77b26db66c9959b631 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:54.880214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:54.881774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:54.895417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8e0a0-4243-42e0-bfae-a6dca00a57fe admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:54.956577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:54.957227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:54.957606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:54.957912 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] PATH_INFO: `/v3/users/01b3240fd64f4e519f6e14f9cab972ca` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:54.959185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b3240fd64f4e519f6e14f9cab972ca)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:54.961166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:54.973780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:55.059203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b3240fd64f4e519f6e14f9cab972ca'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.059911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b3240fd64f4e519f6e14f9cab972ca'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.074481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b3240fd64f4e519f6e14f9cab972ca'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.089146 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b3240fd64f4e519f6e14f9cab972ca'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.097908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-124e5d7c-a6e8-4039-b833-3b3045b84916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b3240fd64f4e519f6e14f9cab972ca'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.106254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1074/2174] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:54 2022] DELETE /identity/v3/users/01b3240fd64f4e519f6e14f9cab972ca => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:55.440120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:55.441916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:55.457718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f3039-a62b-4a64-8861-32271f14f8e4 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:55.531314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:55.531951 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:55.532258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:55.532645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] PATH_INFO: `/v3/projects/52a27104ddb445de87953b6a85a41797` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:55.534342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] RBAC: Authorizing `identity:delete_project(project_id=52a27104ddb445de87953b6a85a41797)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:55.536227 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:55.583966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52a27104ddb445de87953b6a85a41797', 'request_id': 'req-e919a251-f726-454b-8d11-05dd71b0c46b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.590416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52a27104ddb445de87953b6a85a41797', 'request_id': 'req-e919a251-f726-454b-8d11-05dd71b0c46b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.597303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52a27104ddb445de87953b6a85a41797', 'request_id': 'req-e919a251-f726-454b-8d11-05dd71b0c46b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.612123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] The token cache is being invalidate because project 52a27104ddb445de87953b6a85a41797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:55.612410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.613036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e919a251-f726-454b-8d11-05dd71b0c46b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:55.614844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1101/2175] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:55 2022] DELETE /identity/v3/projects/52a27104ddb445de87953b6a85a41797 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:55.620727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1897-9223-4fc0-b2ae-cb8d7b7c9350 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:55.621241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1897-9223-4fc0-b2ae-cb8d7b7c9350 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:55.621327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1897-9223-4fc0-b2ae-cb8d7b7c9350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:55.647769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0c8e1897-9223-4fc0-b2ae-cb8d7b7c9350 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:55.648973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0c8e1897-9223-4fc0-b2ae-cb8d7b7c9350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:55.713559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1075/2176] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:55.722722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531aec2-e11c-45bd-8e63-4b1c7f3bfdb1 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:55.723000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531aec2-e11c-45bd-8e63-4b1c7f3bfdb1 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:55.723268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531aec2-e11c-45bd-8e63-4b1c7f3bfdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:55.742852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e531aec2-e11c-45bd-8e63-4b1c7f3bfdb1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:55.744042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e531aec2-e11c-45bd-8e63-4b1c7f3bfdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:55.763820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1102/2177] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:55.768743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1897-9223-4fc0-b2ae-cb8d7b7c9350 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:55.802105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1897-9223-4fc0-b2ae-cb8d7b7c9350 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:55.810109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:55.810535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:55.810850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:55.811373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:55.812500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:55.813026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:55.814814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:55.849920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1076/2178] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:55.854443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531aec2-e11c-45bd-8e63-4b1c7f3bfdb1 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:55.865931 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531aec2-e11c-45bd-8e63-4b1c7f3bfdb1 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:55.870810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5ada3-e44e-40dd-9767-9dad9b634877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:55.871591 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5ada3-e44e-40dd-9767-9dad9b634877 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:55.871712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5ada3-e44e-40dd-9767-9dad9b634877 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:55.872565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5ada3-e44e-40dd-9767-9dad9b634877 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:55.872889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5ada3-e44e-40dd-9767-9dad9b634877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:55.875076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5ada3-e44e-40dd-9767-9dad9b634877 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:55.916786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1103/2179] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:55 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:55.918227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:55.929216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856cdc-fa72-4774-9635-269785c3c39f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:55.989610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a5fc4-653c-4da9-9b91-5ee92cb83954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:55.990215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a5fc4-653c-4da9-9b91-5ee92cb83954 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:55.990491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a5fc4-653c-4da9-9b91-5ee92cb83954 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:55.990761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a5fc4-653c-4da9-9b91-5ee92cb83954 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:55.991382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a5fc4-653c-4da9-9b91-5ee92cb83954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:55.993337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a5fc4-653c-4da9-9b91-5ee92cb83954 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:56.014684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1077/2180] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:55 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:56.019562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5ada3-e44e-40dd-9767-9dad9b634877 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:56.030199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5ada3-e44e-40dd-9767-9dad9b634877 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:56.034826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb657251-e7ea-47bf-b995-f26c85c42e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:56.035424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657251-e7ea-47bf-b995-f26c85c42e85 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.035696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657251-e7ea-47bf-b995-f26c85c42e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.035962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb657251-e7ea-47bf-b995-f26c85c42e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.036627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb657251-e7ea-47bf-b995-f26c85c42e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:56.038213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb657251-e7ea-47bf-b995-f26c85c42e85 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:56.048486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1104/2181] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:56.052654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a5fc4-653c-4da9-9b91-5ee92cb83954 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:56.060342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:56.065176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a5fc4-653c-4da9-9b91-5ee92cb83954 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:56.071464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91168f-693b-4e4d-bf88-10718429ea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:56.072435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91168f-693b-4e4d-bf88-10718429ea3d admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.072730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91168f-693b-4e4d-bf88-10718429ea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.077423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91168f-693b-4e4d-bf88-10718429ea3d admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.078153 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91168f-693b-4e4d-bf88-10718429ea3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:56.079846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91168f-693b-4e4d-bf88-10718429ea3d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:56.091726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1078/2182] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:56.096278 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb657251-e7ea-47bf-b995-f26c85c42e85 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:56.106524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb657251-e7ea-47bf-b995-f26c85c42e85 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:56.111221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:56.111870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.112150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.112424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] PATH_INFO: `/v3/projects/23586882ddab405a8f78d00435c9d764/users/b9979bfd4fe54d67a2b3c6f4526f20af/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.128066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23586882ddab405a8f78d00435c9d764, user_id=b9979bfd4fe54d67a2b3c6f4526f20af, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:56.130860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:56.145537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1105/2183] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:56 2022] PUT /identity/v3/projects/23586882ddab405a8f78d00435c9d764/users/b9979bfd4fe54d67a2b3c6f4526f20af/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:56.150304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309d6ce-dbcc-432c-96d2-92318e05ce5c None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.150581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309d6ce-dbcc-432c-96d2-92318e05ce5c None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.150875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309d6ce-dbcc-432c-96d2-92318e05ce5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.177474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-2309d6ce-dbcc-432c-96d2-92318e05ce5c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:56.178841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2309d6ce-dbcc-432c-96d2-92318e05ce5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:56.237721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1079/2184] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:56.247502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:56.249233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:56.263318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:56.310258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d9ce9-d4dd-44b0-a804-60f263d6df08 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:56.313884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:56.315020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.315408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.315706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.326207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:56.327619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:56.336939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1106/2185] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:56.539767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107788e-39fd-4e7d-a0b6-05897bcb2aea None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.539767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107788e-39fd-4e7d-a0b6-05897bcb2aea None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.540257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107788e-39fd-4e7d-a0b6-05897bcb2aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.563343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-1107788e-39fd-4e7d-a0b6-05897bcb2aea None None] MFA Rules not processed for user `6f366d6ae3b4404f9fa4343913f67998`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:56.565680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1107788e-39fd-4e7d-a0b6-05897bcb2aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:56.613732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:56.615261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:56.629202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66109305-41a7-4447-8dda-c4559fc48afc service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:56.635194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:56.635843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.636144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.636416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] PATH_INFO: `/v3/users/385fbed74cd74f57bb4a760673b44e47` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.637763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] RBAC: Authorizing `identity:delete_user(user_id=385fbed74cd74f57bb4a760673b44e47)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:56.639706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:56.653348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1080/2186] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:56.711084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '385fbed74cd74f57bb4a760673b44e47'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:56.711751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '385fbed74cd74f57bb4a760673b44e47'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:56.736146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '385fbed74cd74f57bb4a760673b44e47'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:56.745025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '385fbed74cd74f57bb4a760673b44e47'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:56.745682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '385fbed74cd74f57bb4a760673b44e47'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:56.758262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1107/2187] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:56 2022] DELETE /identity/v3/users/385fbed74cd74f57bb4a760673b44e47 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:56.762896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2fc09-b760-44f6-9fc6-8e5f67377d03 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.763263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2fc09-b760-44f6-9fc6-8e5f67377d03 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.763545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2fc09-b760-44f6-9fc6-8e5f67377d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.782810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-70a2fc09-b760-44f6-9fc6-8e5f67377d03 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:56.784546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-70a2fc09-b760-44f6-9fc6-8e5f67377d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:56.843431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1081/2188] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:56.855777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:56.857966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:56.951150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5a0a4-3803-471d-8531-72cb156dd49e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:56.957024 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:56.957982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:56.958543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:56.958817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:56.968852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:56.970224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:56.981351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1108/2189] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:57.116652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce904-b97a-42c2-b533-4abc95e4b9ae None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.116945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce904-b97a-42c2-b533-4abc95e4b9ae None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.117252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce904-b97a-42c2-b533-4abc95e4b9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.147429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-858ce904-b97a-42c2-b533-4abc95e4b9ae None None] MFA Rules not processed for user `b9979bfd4fe54d67a2b3c6f4526f20af`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:57.148801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-858ce904-b97a-42c2-b533-4abc95e4b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:57.188470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:57.236908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1082/2190] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:57.238419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:57.241901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:57.259399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55dfdff-3ee9-4bc9-9a9a-a109ff4b0e66 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:57.265921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333aafc-d955-448c-9c84-187502d79622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:57.266611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333aafc-d955-448c-9c84-187502d79622 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.266900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333aafc-d955-448c-9c84-187502d79622 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.267171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333aafc-d955-448c-9c84-187502d79622 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.268867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333aafc-d955-448c-9c84-187502d79622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:57.279385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333aafc-d955-448c-9c84-187502d79622 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:57.320055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1109/2191] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:57 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:57.332872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ce904-b97a-42c2-b533-4abc95e4b9ae None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:57.348797 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ce904-b97a-42c2-b533-4abc95e4b9ae None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:57.416802 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f0db5-5ecc-4585-b3ad-7feccdbcb771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:57.417547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f0db5-5ecc-4585-b3ad-7feccdbcb771 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.417727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f0db5-5ecc-4585-b3ad-7feccdbcb771 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.417985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f0db5-5ecc-4585-b3ad-7feccdbcb771 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.418614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f0db5-5ecc-4585-b3ad-7feccdbcb771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:57.420552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f0db5-5ecc-4585-b3ad-7feccdbcb771 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:57.449441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1083/2192] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:57 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:57.453936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333aafc-d955-448c-9c84-187502d79622 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:57.464982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333aafc-d955-448c-9c84-187502d79622 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:57.470220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:57.470857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.471139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.471416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.472055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:57.473693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:57.475135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:57.488546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1110/2193] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:57.494467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f0db5-5ecc-4585-b3ad-7feccdbcb771 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:57.507169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f0db5-5ecc-4585-b3ad-7feccdbcb771 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:57.512604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64273fba-8d7a-47bb-84e6-e0c8824883f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:57.513333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273fba-8d7a-47bb-84e6-e0c8824883f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.513729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273fba-8d7a-47bb-84e6-e0c8824883f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.514039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64273fba-8d7a-47bb-84e6-e0c8824883f0 admin admin] PATH_INFO: `/v3/projects/bb877040fd814209a2d2a36cf865a89c/users/2bd62cc3ea3043b7a827d778ca4a60ee/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.523946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:57.527483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:57.531753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64273fba-8d7a-47bb-84e6-e0c8824883f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb877040fd814209a2d2a36cf865a89c, user_id=2bd62cc3ea3043b7a827d778ca4a60ee, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:57.534646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64273fba-8d7a-47bb-84e6-e0c8824883f0 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:57.543843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8691e2-fa42-4231-89c7-ae78e7006660 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:57.552300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:57.552951 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.553263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.553538 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] PATH_INFO: `/v3/projects/10d31e5821af4430a93689e6505b9b11` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.554731 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d31e5821af4430a93689e6505b9b11)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:57.555892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1084/2194] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:57 2022] PUT /identity/v3/projects/bb877040fd814209a2d2a36cf865a89c/users/2bd62cc3ea3043b7a827d778ca4a60ee/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:57.558887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:57.559991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64273fba-8d7a-47bb-84e6-e0c8824883f0 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:57.610399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d31e5821af4430a93689e6505b9b11', 'request_id': 'req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.616961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d31e5821af4430a93689e6505b9b11', 'request_id': 'req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.623117 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d31e5821af4430a93689e6505b9b11', 'request_id': 'req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.636173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64273fba-8d7a-47bb-84e6-e0c8824883f0 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:57.636449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] The token cache is being invalidate because project 10d31e5821af4430a93689e6505b9b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:57.636693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.637390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.638889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1111/2195] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:57 2022] DELETE /identity/v3/projects/10d31e5821af4430a93689e6505b9b11 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:57.643286 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:57.644566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:57.682228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:57.701595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:57.702210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.702486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.702757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.703400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:57.705622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:57.715632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1085/2196] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:49:57.719200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01e1ed-572f-4d9a-a2a0-f69edd6738d6 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:57.719737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:57.721532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:57.724127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:57.724778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.725050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.725350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] PATH_INFO: `/v3/users/3dff49e6f1db4fd58b49e5aabdccf38d` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.726451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dff49e6f1db4fd58b49e5aabdccf38d)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:57.728336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:57.733378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a047c-019d-40d9-8bd8-b17b3b6110ca admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:57.738469 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:57.739152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:57.739746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:57.739746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] PATH_INFO: `/v3/projects/bb877040fd814209a2d2a36cf865a89c/users/2bd62cc3ea3043b7a827d778ca4a60ee/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:57.741864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb877040fd814209a2d2a36cf865a89c, user_id=2bd62cc3ea3043b7a827d778ca4a60ee, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:57.744481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:57.757329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1086/2197] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:57 2022] PUT /identity/v3/projects/bb877040fd814209a2d2a36cf865a89c/users/2bd62cc3ea3043b7a827d778ca4a60ee/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:57.787829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dff49e6f1db4fd58b49e5aabdccf38d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.788571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dff49e6f1db4fd58b49e5aabdccf38d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.805083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dff49e6f1db4fd58b49e5aabdccf38d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.814524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dff49e6f1db4fd58b49e5aabdccf38d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.815876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4e72da3d-f73c-47a8-a693-2730095f465e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dff49e6f1db4fd58b49e5aabdccf38d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:57.824554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:57.830758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1112/2198] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:57 2022] DELETE /identity/v3/users/3dff49e6f1db4fd58b49e5aabdccf38d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:49:58.302723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:58.306328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:58.413438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de3b19-fcfd-47e2-b37d-2956dd292ce0 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:58.473965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:58.474522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:58.474801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:58.475073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] PATH_INFO: `/v3/projects/fcaf7d9a0aac4cfda8badbdd9848e12e` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:58.477887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcaf7d9a0aac4cfda8badbdd9848e12e)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:58.480004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:58.559161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcaf7d9a0aac4cfda8badbdd9848e12e', 'request_id': 'req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.563799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e174-829a-4a8e-ae2c-0509908c5425 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:58.569894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e174-829a-4a8e-ae2c-0509908c5425 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:58.570287 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17e174-829a-4a8e-ae2c-0509908c5425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:58.570754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcaf7d9a0aac4cfda8badbdd9848e12e', 'request_id': 'req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.578706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcaf7d9a0aac4cfda8badbdd9848e12e', 'request_id': 'req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.602534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] The token cache is being invalidate because project fcaf7d9a0aac4cfda8badbdd9848e12e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:58.602952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.603955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.609887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1087/2199] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:58 2022] DELETE /identity/v3/projects/fcaf7d9a0aac4cfda8badbdd9848e12e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:58.611877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:58.615295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:58.622899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:58.628227 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-0a17e174-829a-4a8e-ae2c-0509908c5425 None None] MFA Rules not processed for user `2bd62cc3ea3043b7a827d778ca4a60ee`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:58.631439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0a17e174-829a-4a8e-ae2c-0509908c5425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:58.719921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c4b98-f3e7-43f7-854a-b20b0ebd96ea admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:58.727575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:58.728821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:58.729343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:58.729837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] PATH_INFO: `/v3/users/a7c8cdfc7ec74b10972bdc28d455dba3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:58.731485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7c8cdfc7ec74b10972bdc28d455dba3)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:58.735132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:58.735875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1113/2200] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:58.757161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17e174-829a-4a8e-ae2c-0509908c5425 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:58.757161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0a17e174-829a-4a8e-ae2c-0509908c5425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:58.829824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7c8cdfc7ec74b10972bdc28d455dba3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.831000 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7c8cdfc7ec74b10972bdc28d455dba3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.844982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7c8cdfc7ec74b10972bdc28d455dba3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.854163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17e174-829a-4a8e-ae2c-0509908c5425 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:58.858234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00532791-7541-438e-8a34-5b1e99e97629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:58.858497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7c8cdfc7ec74b10972bdc28d455dba3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.858974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532791-7541-438e-8a34-5b1e99e97629 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:58.860062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532791-7541-438e-8a34-5b1e99e97629 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:58.860062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00532791-7541-438e-8a34-5b1e99e97629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:58.862596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-00532791-7541-438e-8a34-5b1e99e97629 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:58.866572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7c8cdfc7ec74b10972bdc28d455dba3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:58.873732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1088/2201] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:49:58 2022] DELETE /identity/v3/users/a7c8cdfc7ec74b10972bdc28d455dba3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:58.939658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00532791-7541-438e-8a34-5b1e99e97629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:58.941133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00532791-7541-438e-8a34-5b1e99e97629 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:58.958735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1114/2202] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:49:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:49:59.305951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:59.309959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:59.450011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a14c15-5100-4437-8e61-94c2461d24f1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:59.457874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:59.458531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:59.458809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:59.459082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] PATH_INFO: `/v3/projects/04f9ac1918c040f19061b7a36a46878d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:59.460247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f9ac1918c040f19061b7a36a46878d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:59.463772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:59.525658 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:59.551799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f9ac1918c040f19061b7a36a46878d', 'request_id': 'req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:59.557380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f9ac1918c040f19061b7a36a46878d', 'request_id': 'req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:59.563501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f9ac1918c040f19061b7a36a46878d', 'request_id': 'req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:59.576279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] The token cache is being invalidate because project 04f9ac1918c040f19061b7a36a46878d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:49:59.576569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:59.577232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1dfabf7d-751d-4e8f-92a1-226b3b1ea274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:49:59.578708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1089/2203] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:49:59 2022] DELETE /identity/v3/projects/04f9ac1918c040f19061b7a36a46878d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:49:59.585354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49eda3-5e1a-4964-a1a1-ef49552760c8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:59.585638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49eda3-5e1a-4964-a1a1-ef49552760c8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:59.585913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49eda3-5e1a-4964-a1a1-ef49552760c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:59.606271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-fe49eda3-5e1a-4964-a1a1-ef49552760c8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:59.607811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-fe49eda3-5e1a-4964-a1a1-ef49552760c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:59.643851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:49:59.670816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1115/2204] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:59.679099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ba556-f873-4fb9-8349-9e198013dc1a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:59.679491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ba556-f873-4fb9-8349-9e198013dc1a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:59.679821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ba556-f873-4fb9-8349-9e198013dc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:59.699747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-cb4ba556-f873-4fb9-8349-9e198013dc1a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:49:59.700936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cb4ba556-f873-4fb9-8349-9e198013dc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:49:59.709426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1090/2205] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:49:59.713641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49eda3-5e1a-4964-a1a1-ef49552760c8 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:59.723118 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49eda3-5e1a-4964-a1a1-ef49552760c8 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:59.727685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:59.728279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:59.728556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:59.728822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:59.729508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:49:59.729809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:59.731373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:59.742667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1116/2206] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:49:59.747306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ba556-f873-4fb9-8349-9e198013dc1a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:59.756915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ba556-f873-4fb9-8349-9e198013dc1a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:59.761654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f636f73-d604-48b2-9dee-491737951e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:59.762241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f636f73-d604-48b2-9dee-491737951e1c admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:59.762517 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f636f73-d604-48b2-9dee-491737951e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:59.762785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f636f73-d604-48b2-9dee-491737951e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:59.763403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f636f73-d604-48b2-9dee-491737951e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:59.765443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f636f73-d604-48b2-9dee-491737951e1c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:59.792710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1091/2207] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:49:59 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:49:59.797109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:59.807544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5328e-ce7c-49b8-8c3f-4ce3efc54b53 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:59.870391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765042d1-a8e7-4566-b27b-44d7e76307b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:59.871019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765042d1-a8e7-4566-b27b-44d7e76307b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:59.871296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765042d1-a8e7-4566-b27b-44d7e76307b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:59.871559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765042d1-a8e7-4566-b27b-44d7e76307b2 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:59.872583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765042d1-a8e7-4566-b27b-44d7e76307b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:59.874825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765042d1-a8e7-4566-b27b-44d7e76307b2 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:59.910605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1117/2208] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:49:59 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:49:59.914817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f636f73-d604-48b2-9dee-491737951e1c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:59.931562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f636f73-d604-48b2-9dee-491737951e1c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:59.936860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281891-a3a9-4d39-bbc1-f544980f34c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:59.937484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281891-a3a9-4d39-bbc1-f544980f34c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:59.937762 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281891-a3a9-4d39-bbc1-f544980f34c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:59.938031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd281891-a3a9-4d39-bbc1-f544980f34c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:59.938702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd281891-a3a9-4d39-bbc1-f544980f34c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:59.940860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd281891-a3a9-4d39-bbc1-f544980f34c1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:59.951482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1092/2209] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:59.956234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765042d1-a8e7-4566-b27b-44d7e76307b2 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:49:59.967045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765042d1-a8e7-4566-b27b-44d7e76307b2 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:49:59.975177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c989c-f9ff-47ac-b76a-f10f279fd4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:49:59.976070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c989c-f9ff-47ac-b76a-f10f279fd4eb admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:49:59.976449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c989c-f9ff-47ac-b76a-f10f279fd4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:49:59.976828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c989c-f9ff-47ac-b76a-f10f279fd4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:49:59.977860 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c989c-f9ff-47ac-b76a-f10f279fd4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:49:59.979932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c989c-f9ff-47ac-b76a-f10f279fd4eb admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:49:59.992425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1118/2210] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:49:59.996680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281891-a3a9-4d39-bbc1-f544980f34c1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.007707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd281891-a3a9-4d39-bbc1-f544980f34c1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.012456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a55eb-c5df-40e7-9c10-27599ddd153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.013135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a55eb-c5df-40e7-9c10-27599ddd153c admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.013420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a55eb-c5df-40e7-9c10-27599ddd153c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.013697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a55eb-c5df-40e7-9c10-27599ddd153c admin admin] PATH_INFO: `/v3/projects/1004dda6a62c48e2a6e62e913b575edc/users/a8f87f90829540b0bcd2323f70669012/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.030229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a55eb-c5df-40e7-9c10-27599ddd153c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1004dda6a62c48e2a6e62e913b575edc, user_id=a8f87f90829540b0bcd2323f70669012, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.033512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a55eb-c5df-40e7-9c10-27599ddd153c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.046185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1093/2211] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:49:59 2022] PUT /identity/v3/projects/1004dda6a62c48e2a6e62e913b575edc/users/a8f87f90829540b0bcd2323f70669012/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:50:00.057775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d9bf-906a-4633-b38e-c6fda9dd7305 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.058061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d9bf-906a-4633-b38e-c6fda9dd7305 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.058332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d9bf-906a-4633-b38e-c6fda9dd7305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.089205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9a06d9bf-906a-4633-b38e-c6fda9dd7305 None None] MFA Rules not processed for user `a8f87f90829540b0bcd2323f70669012`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:00.091393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9a06d9bf-906a-4633-b38e-c6fda9dd7305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:00.174782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1119/2212] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:00.179339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a55eb-c5df-40e7-9c10-27599ddd153c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.248318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a55eb-c5df-40e7-9c10-27599ddd153c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.253745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ef54a-390b-42b5-aa98-dbf71291c1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.254429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ef54a-390b-42b5-aa98-dbf71291c1cc admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.254714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ef54a-390b-42b5-aa98-dbf71291c1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.254988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ef54a-390b-42b5-aa98-dbf71291c1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.255621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ef54a-390b-42b5-aa98-dbf71291c1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.257552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ef54a-390b-42b5-aa98-dbf71291c1cc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.284244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1094/2213] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:50:00 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:50:00.288280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d9bf-906a-4633-b38e-c6fda9dd7305 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.299352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d9bf-906a-4633-b38e-c6fda9dd7305 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.330159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:00.357538 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b2e85-eab5-4489-b181-78e59a123b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.358228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b2e85-eab5-4489-b181-78e59a123b33 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.358508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b2e85-eab5-4489-b181-78e59a123b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.358782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b2e85-eab5-4489-b181-78e59a123b33 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.359416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b2e85-eab5-4489-b181-78e59a123b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.361993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b2e85-eab5-4489-b181-78e59a123b33 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.385984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1120/2214] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:50:00 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:50:00.387046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ef54a-390b-42b5-aa98-dbf71291c1cc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.397779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ef54a-390b-42b5-aa98-dbf71291c1cc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.402578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91429194-2718-4267-b856-d86e022d96bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.403252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91429194-2718-4267-b856-d86e022d96bf admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.403535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91429194-2718-4267-b856-d86e022d96bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.403810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91429194-2718-4267-b856-d86e022d96bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.404483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91429194-2718-4267-b856-d86e022d96bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.406136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91429194-2718-4267-b856-d86e022d96bf admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.417275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1095/2215] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:50:00.421323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b2e85-eab5-4489-b181-78e59a123b33 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.432634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b2e85-eab5-4489-b181-78e59a123b33 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.437586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dcb85-dc40-4fd3-9901-0731360b2b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.438269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dcb85-dc40-4fd3-9901-0731360b2b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.438556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dcb85-dc40-4fd3-9901-0731360b2b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.438823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dcb85-dc40-4fd3-9901-0731360b2b76 admin admin] PATH_INFO: `/v3/projects/ad9e6f26a0464ad1a1a5ae3078f51bfd/users/38b2cefa013547d9b2aa3e87715445fa/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.455209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dcb85-dc40-4fd3-9901-0731360b2b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9e6f26a0464ad1a1a5ae3078f51bfd, user_id=38b2cefa013547d9b2aa3e87715445fa, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.457912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dcb85-dc40-4fd3-9901-0731360b2b76 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.472381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1121/2216] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:50:00 2022] PUT /identity/v3/projects/ad9e6f26a0464ad1a1a5ae3078f51bfd/users/38b2cefa013547d9b2aa3e87715445fa/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:00.476595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91429194-2718-4267-b856-d86e022d96bf admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.488388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91429194-2718-4267-b856-d86e022d96bf admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.506453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:00.547176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.547782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.548083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.548352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.549642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.551285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.564556 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1096/2217] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:50:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:50:00.565879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dcb85-dc40-4fd3-9901-0731360b2b76 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.577988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.579752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:00.584325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dcb85-dc40-4fd3-9901-0731360b2b76 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.590375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4e640-772a-42ab-b675-64d74d394b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.593399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4e640-772a-42ab-b675-64d74d394b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.593777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4e640-772a-42ab-b675-64d74d394b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.594060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4e640-772a-42ab-b675-64d74d394b2e admin admin] PATH_INFO: `/v3/projects/ad9e6f26a0464ad1a1a5ae3078f51bfd/users/38b2cefa013547d9b2aa3e87715445fa/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.597571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4e640-772a-42ab-b675-64d74d394b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9e6f26a0464ad1a1a5ae3078f51bfd, user_id=38b2cefa013547d9b2aa3e87715445fa, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.601551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4e640-772a-42ab-b675-64d74d394b2e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.617407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1122/2218] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:50:00 2022] PUT /identity/v3/projects/ad9e6f26a0464ad1a1a5ae3078f51bfd/users/38b2cefa013547d9b2aa3e87715445fa/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:00.628651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b22e6-123a-4b2e-9c66-294501007177 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.629011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b22e6-123a-4b2e-9c66-294501007177 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.629389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b22e6-123a-4b2e-9c66-294501007177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.657568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d5e20-4f0f-4e2a-8e49-a7a3ab7d26e7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.666773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-db0b22e6-123a-4b2e-9c66-294501007177 None None] MFA Rules not processed for user `38b2cefa013547d9b2aa3e87715445fa`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:00.668075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-db0b22e6-123a-4b2e-9c66-294501007177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:00.716017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.716899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.717272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.717555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.719630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:00.764952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1123/2219] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:00 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:50:00.777037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b22e6-123a-4b2e-9c66-294501007177 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.778707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-db0b22e6-123a-4b2e-9c66-294501007177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:00.793423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b22e6-123a-4b2e-9c66-294501007177 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.799426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6d59b-71fb-499e-a99c-a3b3cd8fb9a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.800526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d59b-71fb-499e-a99c-a3b3cd8fb9a9 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.801168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.801651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d59b-71fb-499e-a99c-a3b3cd8fb9a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.801651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6d59b-71fb-499e-a99c-a3b3cd8fb9a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.802748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.815684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1097/2220] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:50:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:50:00.828827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6d59b-71fb-499e-a99c-a3b3cd8fb9a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:00.830350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6d59b-71fb-499e-a99c-a3b3cd8fb9a9 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:00.840514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1124/2221] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:50:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:50:00.926165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:00.942254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062681b-22c2-40b4-a639-bec9444713fc service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:00.946430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f51af-77dc-4338-9dbe-902ef62239a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:00.947181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f51af-77dc-4338-9dbe-902ef62239a4 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:00.947535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f51af-77dc-4338-9dbe-902ef62239a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:00.947870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f51af-77dc-4338-9dbe-902ef62239a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:00.984462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:01.036313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f51af-77dc-4338-9dbe-902ef62239a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:01.037899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f51af-77dc-4338-9dbe-902ef62239a4 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:01.049841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1098/2222] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:50:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:50:01.324002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6d59b-71fb-499e-a99c-a3b3cd8fb9a9 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:01.382932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:01.387402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6d59b-71fb-499e-a99c-a3b3cd8fb9a9 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:01.393438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:01.394281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:01.394566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:01.394837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] PATH_INFO: `/v3/users/a8f87f90829540b0bcd2323f70669012` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:01.396155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f87f90829540b0bcd2323f70669012)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:01.398497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:01.445923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f87f90829540b0bcd2323f70669012'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:01.446649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f87f90829540b0bcd2323f70669012'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:01.459868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f87f90829540b0bcd2323f70669012'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:01.466764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f87f90829540b0bcd2323f70669012'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:01.467375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f87f90829540b0bcd2323f70669012'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:01.480145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1125/2223] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:50:01 2022] DELETE /identity/v3/users/a8f87f90829540b0bcd2323f70669012 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:01.484537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84fabc-8b99-4d7b-8b51-82e7fa3ccba2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:01.484822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84fabc-8b99-4d7b-8b51-82e7fa3ccba2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:01.485150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84fabc-8b99-4d7b-8b51-82e7fa3ccba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:01.503754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7e84fabc-8b99-4d7b-8b51-82e7fa3ccba2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:01.504960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7e84fabc-8b99-4d7b-8b51-82e7fa3ccba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:01.565959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1099/2224] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:01.575681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:01.577112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:01.633838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc9cdf-7dca-431f-b049-61a29b8701ca admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:01.637523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd438eb-cb0e-4878-8488-bb38a37dae97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:01.638123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd438eb-cb0e-4878-8488-bb38a37dae97 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:01.638401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd438eb-cb0e-4878-8488-bb38a37dae97 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:01.638681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd438eb-cb0e-4878-8488-bb38a37dae97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:01.648122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd438eb-cb0e-4878-8488-bb38a37dae97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:01.649577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd438eb-cb0e-4878-8488-bb38a37dae97 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:01.659437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1126/2225] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:50:01.956702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84fabc-8b99-4d7b-8b51-82e7fa3ccba2 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:01.958770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7e84fabc-8b99-4d7b-8b51-82e7fa3ccba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:01.978434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84fabc-8b99-4d7b-8b51-82e7fa3ccba2 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:01.984885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:01.986150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:01.986771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:01.986855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] PATH_INFO: `/v3/projects/1004dda6a62c48e2a6e62e913b575edc` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:01.988175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1004dda6a62c48e2a6e62e913b575edc)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:01.990195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:02.041177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1004dda6a62c48e2a6e62e913b575edc', 'request_id': 'req-384a9241-87aa-4a14-8952-b027fa24f8ea'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.046670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1004dda6a62c48e2a6e62e913b575edc', 'request_id': 'req-384a9241-87aa-4a14-8952-b027fa24f8ea'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.052989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1004dda6a62c48e2a6e62e913b575edc', 'request_id': 'req-384a9241-87aa-4a14-8952-b027fa24f8ea'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.067060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] The token cache is being invalidate because project 1004dda6a62c48e2a6e62e913b575edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:50:02.067294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.067942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.069577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1100/2226] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:50:01 2022] DELETE /identity/v3/projects/1004dda6a62c48e2a6e62e913b575edc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:02.074292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd438eb-cb0e-4878-8488-bb38a37dae97 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:02.086136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd438eb-cb0e-4878-8488-bb38a37dae97 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:02.147003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:02.147639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:02.147917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:02.148196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] PATH_INFO: `/v3/users/38b2cefa013547d9b2aa3e87715445fa` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:02.150441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b2cefa013547d9b2aa3e87715445fa)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:02.152344 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:02.182881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:02.223988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b2cefa013547d9b2aa3e87715445fa'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.224963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b2cefa013547d9b2aa3e87715445fa'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.239525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b2cefa013547d9b2aa3e87715445fa'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.254163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b2cefa013547d9b2aa3e87715445fa'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.254919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-14e6f3c8-f367-4f8f-8183-6a194ec564df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b2cefa013547d9b2aa3e87715445fa'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.279292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1127/2227] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:50:02 2022] DELETE /identity/v3/users/38b2cefa013547d9b2aa3e87715445fa => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:02.623194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:02.625207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:02.642513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a9241-87aa-4a14-8952-b027fa24f8ea admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:02.693576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:02.708161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:02.708794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:02.709099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:02.709378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] PATH_INFO: `/v3/projects/ad9e6f26a0464ad1a1a5ae3078f51bfd` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:02.710572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9e6f26a0464ad1a1a5ae3078f51bfd)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:02.712498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:02.760433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9e6f26a0464ad1a1a5ae3078f51bfd', 'request_id': 'req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.765851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9e6f26a0464ad1a1a5ae3078f51bfd', 'request_id': 'req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.772151 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9e6f26a0464ad1a1a5ae3078f51bfd', 'request_id': 'req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.784473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] The token cache is being invalidate because project ad9e6f26a0464ad1a1a5ae3078f51bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:50:02.784761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.785443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-4ba3ef93-aeac-452f-b9ee-f6bd1015c96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:02.786974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1101/2228] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:50:02 2022] DELETE /identity/v3/projects/ad9e6f26a0464ad1a1a5ae3078f51bfd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:02.792755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407ffb2-dd5a-4907-869a-764c263bdb0c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:02.793037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407ffb2-dd5a-4907-869a-764c263bdb0c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:02.793342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407ffb2-dd5a-4907-869a-764c263bdb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:02.813713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7407ffb2-dd5a-4907-869a-764c263bdb0c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:02.814939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7407ffb2-dd5a-4907-869a-764c263bdb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:02.877977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1128/2229] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:02.885852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da917230-7bda-43ea-97e0-c708841fddfb None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:02.886138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da917230-7bda-43ea-97e0-c708841fddfb None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:02.886417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da917230-7bda-43ea-97e0-c708841fddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:02.906013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-da917230-7bda-43ea-97e0-c708841fddfb None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:02.907232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-da917230-7bda-43ea-97e0-c708841fddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:02.915741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1102/2230] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:02.920000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407ffb2-dd5a-4907-869a-764c263bdb0c None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:02.930542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407ffb2-dd5a-4907-869a-764c263bdb0c None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:02.935182 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:02.935765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:02.936036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:02.936300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:02.936928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:50:02.937261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:02.938833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:02.950136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1129/2231] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:50:02.955192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da917230-7bda-43ea-97e0-c708841fddfb None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:02.964911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da917230-7bda-43ea-97e0-c708841fddfb None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:02.969592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2284b74-6840-4148-8bf2-29db422a7cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:02.970186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2284b74-6840-4148-8bf2-29db422a7cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:02.971013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2284b74-6840-4148-8bf2-29db422a7cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:02.971282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2284b74-6840-4148-8bf2-29db422a7cb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:02.971920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2284b74-6840-4148-8bf2-29db422a7cb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:02.973826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2284b74-6840-4148-8bf2-29db422a7cb4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:03.001700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1103/2232] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:50:02 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:50:03.005969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:03.019557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fca616-cee2-43b5-98d8-8a3eebf529a2 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:03.077343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15fb06-6639-4e10-b474-615f074c549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:03.078032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15fb06-6639-4e10-b474-615f074c549a admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:03.078225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15fb06-6639-4e10-b474-615f074c549a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:03.078494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15fb06-6639-4e10-b474-615f074c549a admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:03.079128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15fb06-6639-4e10-b474-615f074c549a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:03.081846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15fb06-6639-4e10-b474-615f074c549a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:03.105171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1130/2233] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:50:03 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:50:03.110328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2284b74-6840-4148-8bf2-29db422a7cb4 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:03.120828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2284b74-6840-4148-8bf2-29db422a7cb4 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:03.126003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab72abc-519d-42e4-bb1b-60fe8010fc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:03.126641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab72abc-519d-42e4-bb1b-60fe8010fc47 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:03.126865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab72abc-519d-42e4-bb1b-60fe8010fc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:03.127158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab72abc-519d-42e4-bb1b-60fe8010fc47 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:03.127794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab72abc-519d-42e4-bb1b-60fe8010fc47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:03.129469 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab72abc-519d-42e4-bb1b-60fe8010fc47 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:03.144441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1104/2234] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:50:03.145779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15fb06-6639-4e10-b474-615f074c549a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:03.155900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15fb06-6639-4e10-b474-615f074c549a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:03.161595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc118d8-be12-4e2a-aade-cb0b11967acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:03.162261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc118d8-be12-4e2a-aade-cb0b11967acf admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:03.162547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc118d8-be12-4e2a-aade-cb0b11967acf admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:03.162823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc118d8-be12-4e2a-aade-cb0b11967acf admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:03.163467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc118d8-be12-4e2a-aade-cb0b11967acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:03.165041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc118d8-be12-4e2a-aade-cb0b11967acf admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:03.179063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1131/2235] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:50:03.186349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab72abc-519d-42e4-bb1b-60fe8010fc47 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:03.198451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab72abc-519d-42e4-bb1b-60fe8010fc47 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:03.204025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:03.205299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:03.205639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:03.205978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] PATH_INFO: `/v3/projects/b88c8a0deaaa4aba915ef5e2c32a427b/users/56ff8a77507c43fba14f0d997cc912ec/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:03.221515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b88c8a0deaaa4aba915ef5e2c32a427b, user_id=56ff8a77507c43fba14f0d997cc912ec, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:03.224134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:03.237362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1105/2236] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:50:03 2022] PUT /identity/v3/projects/b88c8a0deaaa4aba915ef5e2c32a427b/users/56ff8a77507c43fba14f0d997cc912ec/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:03.247755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a1969-db03-4efa-a565-50e509476e68 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:03.248036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a1969-db03-4efa-a565-50e509476e68 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:03.248301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a1969-db03-4efa-a565-50e509476e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:03.256328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:03.280378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a50a1969-db03-4efa-a565-50e509476e68 None None] MFA Rules not processed for user `56ff8a77507c43fba14f0d997cc912ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:03.282391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a50a1969-db03-4efa-a565-50e509476e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:03.357733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1132/2237] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:03.372921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:03.374667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:03.444515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7c598-daa4-4811-a376-e80175347720 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:03.448933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:03.449808 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:03.450231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:03.450653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:03.465771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:03.469089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:03.488559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1106/2238] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:50:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:50:04.962326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60ecb3-fd2b-40d2-a3eb-bf12738fc2d8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:04.962859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60ecb3-fd2b-40d2-a3eb-bf12738fc2d8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:04.963525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60ecb3-fd2b-40d2-a3eb-bf12738fc2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:04.994850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ad60ecb3-fd2b-40d2-a3eb-bf12738fc2d8 None None] MFA Rules not processed for user `56ff8a77507c43fba14f0d997cc912ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:04.996222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ad60ecb3-fd2b-40d2-a3eb-bf12738fc2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:05.007243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1133/2239] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:50:06.869569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:06.871390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:06.893925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:06.955120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e02d47-243b-4814-855b-a2f6c8174b5e service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:06.960495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:06.961286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:06.961562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:06.961837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] PATH_INFO: `/v3/users/6d415854b71b4366a2f8d11e64271ccc` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:06.962992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d415854b71b4366a2f8d11e64271ccc)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:06.964937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:07.014411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d415854b71b4366a2f8d11e64271ccc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.015099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d415854b71b4366a2f8d11e64271ccc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.023488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d415854b71b4366a2f8d11e64271ccc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.039306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d415854b71b4366a2f8d11e64271ccc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.046470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-206d5755-d505-478d-9e52-494f9e08fd0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d415854b71b4366a2f8d11e64271ccc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.053091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1107/2240] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:50:06 2022] DELETE /identity/v3/users/6d415854b71b4366a2f8d11e64271ccc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:50:07.449301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60ecb3-fd2b-40d2-a3eb-bf12738fc2d8 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:07.462343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60ecb3-fd2b-40d2-a3eb-bf12738fc2d8 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:07.518887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:07.519508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:07.519780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:07.520045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] PATH_INFO: `/v3/projects/940a7e4459874ae3b54c5db3ec096b8d` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:07.521174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940a7e4459874ae3b54c5db3ec096b8d)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:07.523063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:07.570614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940a7e4459874ae3b54c5db3ec096b8d', 'request_id': 'req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.577536 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940a7e4459874ae3b54c5db3ec096b8d', 'request_id': 'req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.584223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940a7e4459874ae3b54c5db3ec096b8d', 'request_id': 'req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.597037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] The token cache is being invalidate because project 940a7e4459874ae3b54c5db3ec096b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:50:07.597363 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.598035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-7a055f91-1dfe-488b-bcbf-5aa13be3fa46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:07.599766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1134/2241] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:50:07 2022] DELETE /identity/v3/projects/940a7e4459874ae3b54c5db3ec096b8d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:07.604553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181227-8d72-406f-9685-c31d5ad02b95 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:07.604905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181227-8d72-406f-9685-c31d5ad02b95 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:07.605299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf181227-8d72-406f-9685-c31d5ad02b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:07.624911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-cf181227-8d72-406f-9685-c31d5ad02b95 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:07.626238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cf181227-8d72-406f-9685-c31d5ad02b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:07.700777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1108/2242] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:07.709165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb733a19-08c3-4ec6-b247-12816223daa9 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:07.709452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb733a19-08c3-4ec6-b247-12816223daa9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:07.709713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb733a19-08c3-4ec6-b247-12816223daa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:07.729834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-bb733a19-08c3-4ec6-b247-12816223daa9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:07.731020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-bb733a19-08c3-4ec6-b247-12816223daa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:07.738985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1135/2243] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:07.743846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf181227-8d72-406f-9685-c31d5ad02b95 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:07.754487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf181227-8d72-406f-9685-c31d5ad02b95 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:07.759092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:07.759751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:07.760092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:07.760427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:07.761159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:50:07.761523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:07.763169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:07.775549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1109/2244] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:50:07.779866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb733a19-08c3-4ec6-b247-12816223daa9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:07.790028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb733a19-08c3-4ec6-b247-12816223daa9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:07.796523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a6b3f-e932-43da-8f58-7d3826ed5941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:07.797593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a6b3f-e932-43da-8f58-7d3826ed5941 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:07.798073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a6b3f-e932-43da-8f58-7d3826ed5941 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:07.798640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a6b3f-e932-43da-8f58-7d3826ed5941 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:07.799330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a6b3f-e932-43da-8f58-7d3826ed5941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:07.801703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a6b3f-e932-43da-8f58-7d3826ed5941 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:07.816313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:07.840210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1136/2245] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:50:07 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:50:07.844468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:07.855873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1ba0e-a68a-43b2-ab0c-5f51651325e8 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:07.883631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:07.937365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7576f4-a302-4e57-bb39-4bc57a08e0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:07.938265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7576f4-a302-4e57-bb39-4bc57a08e0fc admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:07.938538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7576f4-a302-4e57-bb39-4bc57a08e0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:07.939359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7576f4-a302-4e57-bb39-4bc57a08e0fc admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:07.940002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7576f4-a302-4e57-bb39-4bc57a08e0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:07.942815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7576f4-a302-4e57-bb39-4bc57a08e0fc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:07.966997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1110/2246] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:50:07 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:50:07.971364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a6b3f-e932-43da-8f58-7d3826ed5941 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:07.981795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a6b3f-e932-43da-8f58-7d3826ed5941 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:07.987720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b837f34-59b2-48c0-9621-eac4e6548277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:07.988306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b837f34-59b2-48c0-9621-eac4e6548277 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:07.988584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b837f34-59b2-48c0-9621-eac4e6548277 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:07.988855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b837f34-59b2-48c0-9621-eac4e6548277 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:07.989573 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b837f34-59b2-48c0-9621-eac4e6548277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:07.991157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b837f34-59b2-48c0-9621-eac4e6548277 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:08.002291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1137/2247] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:50:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:50:08.006277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7576f4-a302-4e57-bb39-4bc57a08e0fc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:08.016330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7576f4-a302-4e57-bb39-4bc57a08e0fc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:08.021536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966460bd-fe6e-4654-8f47-e75835e91be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:08.022396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966460bd-fe6e-4654-8f47-e75835e91be2 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:08.022747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966460bd-fe6e-4654-8f47-e75835e91be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:08.023168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966460bd-fe6e-4654-8f47-e75835e91be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:08.024050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966460bd-fe6e-4654-8f47-e75835e91be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:08.025686 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966460bd-fe6e-4654-8f47-e75835e91be2 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:08.036864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1111/2248] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:50:08.041099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b837f34-59b2-48c0-9621-eac4e6548277 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:08.052638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b837f34-59b2-48c0-9621-eac4e6548277 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:08.057728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:08.058381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:08.058657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:08.058952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] PATH_INFO: `/v3/projects/37932fe56b864be6b57f92a76e92cd09/users/04b569b91d1045dc8257c8aaa868842a/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:08.073917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=37932fe56b864be6b57f92a76e92cd09, user_id=04b569b91d1045dc8257c8aaa868842a, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:08.076564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:08.089941 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1138/2249] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:50:08 2022] PUT /identity/v3/projects/37932fe56b864be6b57f92a76e92cd09/users/04b569b91d1045dc8257c8aaa868842a/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:08.094987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4560b-3b89-43b0-be0e-105509c78bc9 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:08.095778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4560b-3b89-43b0-be0e-105509c78bc9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:08.096050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4560b-3b89-43b0-be0e-105509c78bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:08.115703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-01b4560b-3b89-43b0-be0e-105509c78bc9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:08.116918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-01b4560b-3b89-43b0-be0e-105509c78bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:08.178890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1112/2250] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:08.189680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:08.191463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:08.250289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5d257-335d-4916-b22a-a38cffdcae4c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:08.253946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:08.254670 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:08.254949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:08.255331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:08.264847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:08.266308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:08.276752 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1139/2251] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:50:14.143507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6cfdd-c4a0-489b-b0d8-4620583c2a67 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:14.144290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6cfdd-c4a0-489b-b0d8-4620583c2a67 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:14.144562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6cfdd-c4a0-489b-b0d8-4620583c2a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:14.185695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8db6cfdd-c4a0-489b-b0d8-4620583c2a67 None None] MFA Rules not processed for user `04b569b91d1045dc8257c8aaa868842a`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:14.187133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8db6cfdd-c4a0-489b-b0d8-4620583c2a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:14.333341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1113/2252] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:14.354918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:14.356470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:14.457718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f876c11-cac8-4bff-be8b-c89fe35ef858 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:14.463751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:14.464452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:14.464796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:14.465180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:14.480010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:14.482014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:14.512511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1140/2253] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:50:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:50:17.043839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee473b-554c-428c-bfa8-009e1a6271a8 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:17.044294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee473b-554c-428c-bfa8-009e1a6271a8 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:17.044371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee473b-554c-428c-bfa8-009e1a6271a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:17.080319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-35ee473b-554c-428c-bfa8-009e1a6271a8 None None] MFA Rules not processed for user `04b569b91d1045dc8257c8aaa868842a`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:17.081570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-35ee473b-554c-428c-bfa8-009e1a6271a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:17.089667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1114/2254] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:17.545271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:17.548851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:17.569558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229e367-31ac-4fda-8a3d-b1045f4aef4a service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:17.578684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:17.580690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:17.581218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:17.581597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] PATH_INFO: `/v3/users/b9979bfd4fe54d67a2b3c6f4526f20af` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:17.583549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9979bfd4fe54d67a2b3c6f4526f20af)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:17.586454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:17.651746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:17.666608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9979bfd4fe54d67a2b3c6f4526f20af'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:17.667425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9979bfd4fe54d67a2b3c6f4526f20af'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:17.682240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9979bfd4fe54d67a2b3c6f4526f20af'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:17.693602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9979bfd4fe54d67a2b3c6f4526f20af'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:17.694491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9979bfd4fe54d67a2b3c6f4526f20af'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:17.709392 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1141/2255] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:50:17 2022] DELETE /identity/v3/users/b9979bfd4fe54d67a2b3c6f4526f20af => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:50:17.950258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee473b-554c-428c-bfa8-009e1a6271a8 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:17.966086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee473b-554c-428c-bfa8-009e1a6271a8 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:18.016340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:18.080077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:18.081542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:18.082050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:18.082538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] PATH_INFO: `/v3/projects/23586882ddab405a8f78d00435c9d764` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:18.083901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23586882ddab405a8f78d00435c9d764)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:18.086064 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:18.142441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23586882ddab405a8f78d00435c9d764', 'request_id': 'req-b374e737-86fa-43b5-ba6b-d36da0ea2a09'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.149461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23586882ddab405a8f78d00435c9d764', 'request_id': 'req-b374e737-86fa-43b5-ba6b-d36da0ea2a09'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.156017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23586882ddab405a8f78d00435c9d764', 'request_id': 'req-b374e737-86fa-43b5-ba6b-d36da0ea2a09'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.185745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] The token cache is being invalidate because project 23586882ddab405a8f78d00435c9d764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:50:18.186468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.188107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.190567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1115/2256] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:50:17 2022] DELETE /identity/v3/projects/23586882ddab405a8f78d00435c9d764 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:18.194593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:18.196087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:18.226927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4c10f-2dd7-45e9-ad91-6f6a2277f03c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:18.387047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:18.387047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:18.387047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:18.387047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] PATH_INFO: `/v3/users/2bd62cc3ea3043b7a827d778ca4a60ee` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:18.388224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd62cc3ea3043b7a827d778ca4a60ee)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:18.388224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:18.460011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd62cc3ea3043b7a827d778ca4a60ee'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.460752 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd62cc3ea3043b7a827d778ca4a60ee'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.486923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd62cc3ea3043b7a827d778ca4a60ee'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.494877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd62cc3ea3043b7a827d778ca4a60ee'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.495861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-12e43864-80af-4a23-aae4-e2a099731abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd62cc3ea3043b7a827d778ca4a60ee'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.525710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1142/2257] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:50:18 2022] DELETE /identity/v3/users/2bd62cc3ea3043b7a827d778ca4a60ee => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:50:18.767744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:18.769408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:18.784337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374e737-86fa-43b5-ba6b-d36da0ea2a09 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:18.853648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:18.855222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:18.855222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:18.855222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] PATH_INFO: `/v3/projects/bb877040fd814209a2d2a36cf865a89c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:18.856670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb877040fd814209a2d2a36cf865a89c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:18.858712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:18.936378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb877040fd814209a2d2a36cf865a89c', 'request_id': 'req-5f3444bc-12f6-4133-92d1-47647986a002'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.941278 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:18.942872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb877040fd814209a2d2a36cf865a89c', 'request_id': 'req-5f3444bc-12f6-4133-92d1-47647986a002'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.951768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb877040fd814209a2d2a36cf865a89c', 'request_id': 'req-5f3444bc-12f6-4133-92d1-47647986a002'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.975882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] The token cache is being invalidate because project bb877040fd814209a2d2a36cf865a89c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:50:18.976328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.977254 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5f3444bc-12f6-4133-92d1-47647986a002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:50:18.979014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1116/2258] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:50:18 2022] DELETE /identity/v3/projects/bb877040fd814209a2d2a36cf865a89c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:18.983733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c12e46-7367-4cb0-aa48-4974e66d914d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:18.984212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c12e46-7367-4cb0-aa48-4974e66d914d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:18.984305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c12e46-7367-4cb0-aa48-4974e66d914d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.007500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-74c12e46-7367-4cb0-aa48-4974e66d914d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:19.008699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-74c12e46-7367-4cb0-aa48-4974e66d914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:19.077279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:19.118877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1143/2259] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:19.126745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d29c8-f937-4217-8dab-2ff5c3b74b2b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.127614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d29c8-f937-4217-8dab-2ff5c3b74b2b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.128044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d29c8-f937-4217-8dab-2ff5c3b74b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.152048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-756d29c8-f937-4217-8dab-2ff5c3b74b2b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:19.153590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-756d29c8-f937-4217-8dab-2ff5c3b74b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:19.175200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1117/2260] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:50:19.180940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c12e46-7367-4cb0-aa48-4974e66d914d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:19.190902 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c12e46-7367-4cb0-aa48-4974e66d914d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:19.196328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:19.196944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.197268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.197525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.198340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:50:19.198785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:19.200374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:19.212328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1144/2261] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:50:19.216797 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d29c8-f937-4217-8dab-2ff5c3b74b2b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:19.226409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d29c8-f937-4217-8dab-2ff5c3b74b2b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:19.231125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66cf3e-7896-45d6-8a33-5e5942392da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:19.231719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66cf3e-7896-45d6-8a33-5e5942392da1 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.231998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66cf3e-7896-45d6-8a33-5e5942392da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.232270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a66cf3e-7896-45d6-8a33-5e5942392da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.232895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66cf3e-7896-45d6-8a33-5e5942392da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:19.234812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a66cf3e-7896-45d6-8a33-5e5942392da1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:19.266963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1118/2262] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:50:19 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:50:19.268373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:19.280113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef3869-750c-44bd-bd4c-4cdb1265dd23 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:19.349486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f45fb-3c2d-4097-afbb-43542b17a8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:19.350398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f45fb-3c2d-4097-afbb-43542b17a8ad admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.350811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f45fb-3c2d-4097-afbb-43542b17a8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.351177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f45fb-3c2d-4097-afbb-43542b17a8ad admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.351905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f45fb-3c2d-4097-afbb-43542b17a8ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:19.353993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f45fb-3c2d-4097-afbb-43542b17a8ad admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:19.380501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1145/2263] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:50:19 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:50:19.384752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66cf3e-7896-45d6-8a33-5e5942392da1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:19.402573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a66cf3e-7896-45d6-8a33-5e5942392da1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:19.408845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f415da-da2c-4e0f-a789-da7efb05873f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:19.409565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f415da-da2c-4e0f-a789-da7efb05873f admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.409922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f415da-da2c-4e0f-a789-da7efb05873f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.410196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f415da-da2c-4e0f-a789-da7efb05873f admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.410978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f415da-da2c-4e0f-a789-da7efb05873f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:19.412649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f415da-da2c-4e0f-a789-da7efb05873f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:19.424041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1119/2264] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:50:19.428104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f45fb-3c2d-4097-afbb-43542b17a8ad admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:19.438560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f45fb-3c2d-4097-afbb-43542b17a8ad admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:19.444764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e2ed3-76d2-4e66-8b8f-385f38187587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:19.445432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e2ed3-76d2-4e66-8b8f-385f38187587 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.445757 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e2ed3-76d2-4e66-8b8f-385f38187587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.445988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e2ed3-76d2-4e66-8b8f-385f38187587 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.446771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e2ed3-76d2-4e66-8b8f-385f38187587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:19.448358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e2ed3-76d2-4e66-8b8f-385f38187587 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:19.459636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1146/2265] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:50:19.463722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f415da-da2c-4e0f-a789-da7efb05873f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:19.474204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f415da-da2c-4e0f-a789-da7efb05873f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:19.478962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:19.479625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.479906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.480177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] PATH_INFO: `/v3/projects/c8297ebd415e47c789c7f12b7e4a0873/users/2d6cdfb617df469f8fa4d55126c0b0b9/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.494636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8297ebd415e47c789c7f12b7e4a0873, user_id=2d6cdfb617df469f8fa4d55126c0b0b9, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:19.497563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:19.511321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1120/2266] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:50:19 2022] PUT /identity/v3/projects/c8297ebd415e47c789c7f12b7e4a0873/users/2d6cdfb617df469f8fa4d55126c0b0b9/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:50:19.522027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f260-bd63-41c8-8708-120ae2f5be73 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.522312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f260-bd63-41c8-8708-120ae2f5be73 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.522577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803f260-bd63-41c8-8708-120ae2f5be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.551343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e803f260-bd63-41c8-8708-120ae2f5be73 None None] MFA Rules not processed for user `2d6cdfb617df469f8fa4d55126c0b0b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:50:19.552587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e803f260-bd63-41c8-8708-120ae2f5be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:19.620258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1147/2267] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:50:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:50:19.633238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:50:19.634980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:50:19.695138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3597d-e3f5-4dad-9863-af261571e6c8 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:50:19.700386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa7e0d-7e1e-4c21-aa01-b7c3d7b2bb59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:50:19.701119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa7e0d-7e1e-4c21-aa01-b7c3d7b2bb59 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:50:19.701402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa7e0d-7e1e-4c21-aa01-b7c3d7b2bb59 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:50:19.701671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa7e0d-7e1e-4c21-aa01-b7c3d7b2bb59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:50:19.710900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa7e0d-7e1e-4c21-aa01-b7c3d7b2bb59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:50:19.712307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa7e0d-7e1e-4c21-aa01-b7c3d7b2bb59 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:50:19.721654 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1121/2268] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:50:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:50:29.616495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:50:29.716958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=58,keystone:INSERT=3,keystone:DELETE=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:04.327603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803f260-bd63-41c8-8708-120ae2f5be73 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:04.329765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e803f260-bd63-41c8-8708-120ae2f5be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:04.394358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803f260-bd63-41c8-8708-120ae2f5be73 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:04.398868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:04.399491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:04.399770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:04.400043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] PATH_INFO: `/v3/users/d99cc43956eb4a709472f39aaf76f488` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:04.401294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d99cc43956eb4a709472f39aaf76f488)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:04.403198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:04.460966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd99cc43956eb4a709472f39aaf76f488'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.461122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd99cc43956eb4a709472f39aaf76f488'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.474539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd99cc43956eb4a709472f39aaf76f488'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.485357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd99cc43956eb4a709472f39aaf76f488'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.486305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd99cc43956eb4a709472f39aaf76f488'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.507726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1148/2269] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:51:04 2022] DELETE /identity/v3/users/d99cc43956eb4a709472f39aaf76f488 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:04.509452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a2012-38b7-416f-b86a-4b19ecb59cc2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:04.509935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a2012-38b7-416f-b86a-4b19ecb59cc2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:04.510113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a2012-38b7-416f-b86a-4b19ecb59cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:04.543848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-cd4a2012-38b7-416f-b86a-4b19ecb59cc2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:04.545238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cd4a2012-38b7-416f-b86a-4b19ecb59cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:04.598873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:04.600766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:04.663014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1122/2270] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:04.677201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a2012-38b7-416f-b86a-4b19ecb59cc2 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:04.678824 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cd4a2012-38b7-416f-b86a-4b19ecb59cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:04.714643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc084fa3-c957-4cc4-a161-1d1d4387ebc8 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:04.720589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:04.721300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:04.721691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:04.722055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] PATH_INFO: `/v3/users/2d6cdfb617df469f8fa4d55126c0b0b9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:04.724008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d6cdfb617df469f8fa4d55126c0b0b9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:04.726564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:04.759449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:04.786352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a2012-38b7-416f-b86a-4b19ecb59cc2 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:04.791245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad0d38-e175-4a6c-9d31-87688f3d6e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:04.792007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad0d38-e175-4a6c-9d31-87688f3d6e00 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:04.792413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad0d38-e175-4a6c-9d31-87688f3d6e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:04.793026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad0d38-e175-4a6c-9d31-87688f3d6e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:04.805555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad0d38-e175-4a6c-9d31-87688f3d6e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:04.807775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad0d38-e175-4a6c-9d31-87688f3d6e00 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:04.823718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d6cdfb617df469f8fa4d55126c0b0b9'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.823718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d6cdfb617df469f8fa4d55126c0b0b9'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.823937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1123/2271] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:04.835293 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d6cdfb617df469f8fa4d55126c0b0b9'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.844188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d6cdfb617df469f8fa4d55126c0b0b9'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.844973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d6cdfb617df469f8fa4d55126c0b0b9'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:04.857913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1149/2272] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:51:04 2022] DELETE /identity/v3/users/2d6cdfb617df469f8fa4d55126c0b0b9 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:04.863858 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2edbcb-f622-4db0-b4d1-1f953349c80e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:04.864154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2edbcb-f622-4db0-b4d1-1f953349c80e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:04.864426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2edbcb-f622-4db0-b4d1-1f953349c80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:04.884283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6f2edbcb-f622-4db0-b4d1-1f953349c80e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:04.885575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6f2edbcb-f622-4db0-b4d1-1f953349c80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:04.957036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1124/2273] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:04.967337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:04.969557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:05.048698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359517-b87a-4813-9daf-426a397c42d0 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.052508 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2af8d6-310b-4f82-a8f4-40f594f86c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.053150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af8d6-310b-4f82-a8f4-40f594f86c47 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.053431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af8d6-310b-4f82-a8f4-40f594f86c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.053756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af8d6-310b-4f82-a8f4-40f594f86c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.066289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2af8d6-310b-4f82-a8f4-40f594f86c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.067711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2af8d6-310b-4f82-a8f4-40f594f86c47 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.077689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1150/2274] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:05.128512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2edbcb-f622-4db0-b4d1-1f953349c80e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.129990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6f2edbcb-f622-4db0-b4d1-1f953349c80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:05.142631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2edbcb-f622-4db0-b4d1-1f953349c80e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.147793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.148569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.149096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.150385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] PATH_INFO: `/v3/projects/58c6ea5863b143e688246dde52e891d2` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.160132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c6ea5863b143e688246dde52e891d2)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.162348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.205907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:05.215094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c6ea5863b143e688246dde52e891d2', 'request_id': 'req-510fe65c-4c5d-4317-8128-872d2ef428eb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.220399 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c6ea5863b143e688246dde52e891d2', 'request_id': 'req-510fe65c-4c5d-4317-8128-872d2ef428eb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.227361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c6ea5863b143e688246dde52e891d2', 'request_id': 'req-510fe65c-4c5d-4317-8128-872d2ef428eb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.243354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] The token cache is being invalidate because project 58c6ea5863b143e688246dde52e891d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:51:05.243958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.244638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-510fe65c-4c5d-4317-8128-872d2ef428eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.246459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1125/2275] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:51:05 2022] DELETE /identity/v3/projects/58c6ea5863b143e688246dde52e891d2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:05.251890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ded3e-6c70-4a07-956c-1428fd1eacb6 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.252175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ded3e-6c70-4a07-956c-1428fd1eacb6 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.252406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ded3e-6c70-4a07-956c-1428fd1eacb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.273126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-7f4ded3e-6c70-4a07-956c-1428fd1eacb6 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:05.274353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7f4ded3e-6c70-4a07-956c-1428fd1eacb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:05.361544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1151/2276] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:05.366629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d083415-5fd2-4756-bd78-a238d3fb2022 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.366884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d083415-5fd2-4756-bd78-a238d3fb2022 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.367157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d083415-5fd2-4756-bd78-a238d3fb2022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.389671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9d083415-5fd2-4756-bd78-a238d3fb2022 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:05.391875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9d083415-5fd2-4756-bd78-a238d3fb2022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:05.404660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1126/2277] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:05.410209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ded3e-6c70-4a07-956c-1428fd1eacb6 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.420585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d083415-5fd2-4756-bd78-a238d3fb2022 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.420758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ded3e-6c70-4a07-956c-1428fd1eacb6 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.422183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9d083415-5fd2-4756-bd78-a238d3fb2022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:05.435752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d083415-5fd2-4756-bd78-a238d3fb2022 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.440941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.441701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.442030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.442308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] PATH_INFO: `/v3/projects/c8297ebd415e47c789c7f12b7e4a0873` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.443547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8297ebd415e47c789c7f12b7e4a0873)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.445570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.447312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.448367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.449108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.449601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.450452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:51:05.451011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.455017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.469785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1152/2278] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:51:05.471427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.481829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19539970-2bd4-4514-84e7-5a88b202a8c0 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.486510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10345652-129b-418e-8635-60e22d6c7486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.487103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10345652-129b-418e-8635-60e22d6c7486 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.487379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10345652-129b-418e-8635-60e22d6c7486 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.487679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10345652-129b-418e-8635-60e22d6c7486 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.488369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10345652-129b-418e-8635-60e22d6c7486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.490281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10345652-129b-418e-8635-60e22d6c7486 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.498148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8297ebd415e47c789c7f12b7e4a0873', 'request_id': 'req-d450e2e5-02fc-4030-abfa-1b7a07157cbf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.505175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8297ebd415e47c789c7f12b7e4a0873', 'request_id': 'req-d450e2e5-02fc-4030-abfa-1b7a07157cbf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.511581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8297ebd415e47c789c7f12b7e4a0873', 'request_id': 'req-d450e2e5-02fc-4030-abfa-1b7a07157cbf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.522526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1153/2279] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:51:05 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:05.523830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10345652-129b-418e-8635-60e22d6c7486 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.525812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] The token cache is being invalidate because project c8297ebd415e47c789c7f12b7e4a0873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:51:05.526090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.527327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:05.529261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1127/2280] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:51:05 2022] DELETE /identity/v3/projects/c8297ebd415e47c789c7f12b7e4a0873 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:05.539027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:05.601502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10345652-129b-418e-8635-60e22d6c7486 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.606471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415cfe7-716e-4d49-b0b1-3cddd199951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.607138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415cfe7-716e-4d49-b0b1-3cddd199951d admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.607415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415cfe7-716e-4d49-b0b1-3cddd199951d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.607686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415cfe7-716e-4d49-b0b1-3cddd199951d admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.608337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415cfe7-716e-4d49-b0b1-3cddd199951d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.610403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415cfe7-716e-4d49-b0b1-3cddd199951d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.633432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1154/2281] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:51:05 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:51:05.637913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.639483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:05.708190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450e2e5-02fc-4030-abfa-1b7a07157cbf admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.712930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d2a5b-2b89-46ce-a27e-ba61c3916635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.713576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d2a5b-2b89-46ce-a27e-ba61c3916635 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.713852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d2a5b-2b89-46ce-a27e-ba61c3916635 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.714117 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d2a5b-2b89-46ce-a27e-ba61c3916635 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.714796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d2a5b-2b89-46ce-a27e-ba61c3916635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.716413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d2a5b-2b89-46ce-a27e-ba61c3916635 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.730666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1128/2282] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:51:05.731911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415cfe7-716e-4d49-b0b1-3cddd199951d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.742795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415cfe7-716e-4d49-b0b1-3cddd199951d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.747721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bed89-91d7-4929-bfc7-001673027491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.748312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bed89-91d7-4929-bfc7-001673027491 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.748588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bed89-91d7-4929-bfc7-001673027491 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.748859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bed89-91d7-4929-bfc7-001673027491 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.749530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bed89-91d7-4929-bfc7-001673027491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.751103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bed89-91d7-4929-bfc7-001673027491 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.760869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1155/2283] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:51:05.764998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d2a5b-2b89-46ce-a27e-ba61c3916635 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.775166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d2a5b-2b89-46ce-a27e-ba61c3916635 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.779689 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.780341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.780594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.780860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] PATH_INFO: `/v3/projects/d7b1df9340cd470485c850ec89938b87/users/425e9db5848940769958042e579be565/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.796796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b1df9340cd470485c850ec89938b87, user_id=425e9db5848940769958042e579be565, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.799999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.824380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1129/2284] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:51:05 2022] PUT /identity/v3/projects/d7b1df9340cd470485c850ec89938b87/users/425e9db5848940769958042e579be565/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:05.826831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d738dde-34bf-4480-990f-cc3d3b032e38 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.826831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d738dde-34bf-4480-990f-cc3d3b032e38 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.826831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d738dde-34bf-4480-990f-cc3d3b032e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.835287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.837548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:05.861900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:05.881477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-0d738dde-34bf-4480-990f-cc3d3b032e38 None None] MFA Rules not processed for user `425e9db5848940769958042e579be565`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:05.883933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0d738dde-34bf-4480-990f-cc3d3b032e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:05.940685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca8b7-2fa5-4eff-97a5-abb56becd7db admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.945536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.946173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.946453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.946827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] PATH_INFO: `/v3/users/5bd8f221468f4ca18c0f6c13a6d3b393` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.947842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd8f221468f4ca18c0f6c13a6d3b393)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.949922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:05.964099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1156/2285] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:05.969221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d738dde-34bf-4480-990f-cc3d3b032e38 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:05.983570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d738dde-34bf-4480-990f-cc3d3b032e38 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:05.989440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9942f61-6edf-41f1-ba3f-6f83a2146407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:05.990141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9942f61-6edf-41f1-ba3f-6f83a2146407 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:05.990445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9942f61-6edf-41f1-ba3f-6f83a2146407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:05.990854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9942f61-6edf-41f1-ba3f-6f83a2146407 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:05.991502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9942f61-6edf-41f1-ba3f-6f83a2146407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:05.993451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9942f61-6edf-41f1-ba3f-6f83a2146407 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.022354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd8f221468f4ca18c0f6c13a6d3b393'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:06.023171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd8f221468f4ca18c0f6c13a6d3b393'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:06.034380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd8f221468f4ca18c0f6c13a6d3b393'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:06.038460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1157/2286] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:51:05 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:06.042657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9942f61-6edf-41f1-ba3f-6f83a2146407 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.051887 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd8f221468f4ca18c0f6c13a6d3b393'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:06.061157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-78d6259c-2240-4593-a576-d9deb453cf7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd8f221468f4ca18c0f6c13a6d3b393'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:06.071593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1130/2287] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:51:05 2022] DELETE /identity/v3/users/5bd8f221468f4ca18c0f6c13a6d3b393 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:51:06.073114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e62fc-904a-4445-bea3-e694d3d2e527 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.073378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e62fc-904a-4445-bea3-e694d3d2e527 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.073747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e62fc-904a-4445-bea3-e694d3d2e527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.095045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-528e62fc-904a-4445-bea3-e694d3d2e527 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:06.096330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-528e62fc-904a-4445-bea3-e694d3d2e527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:06.108231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:06.134748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9942f61-6edf-41f1-ba3f-6f83a2146407 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.176989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1131/2288] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:06.186625 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e62fc-904a-4445-bea3-e694d3d2e527 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.188554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-528e62fc-904a-4445-bea3-e694d3d2e527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:06.215296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.216065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.216833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.217217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.218282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.220900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.231778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:06.247753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1158/2289] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:51:06 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:51:06.251405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.253078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:06.254420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e62fc-904a-4445-bea3-e694d3d2e527 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.258670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545b2ac-6fd7-4fff-8f15-77892dfe2adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.259365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b2ac-6fd7-4fff-8f15-77892dfe2adb service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.259645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b2ac-6fd7-4fff-8f15-77892dfe2adb service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.260221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545b2ac-6fd7-4fff-8f15-77892dfe2adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.264862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d44e3-3772-436e-b0b9-1b12dc43842b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.270107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8e6de-ff57-42b3-bb42-d54fd3d611c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.270705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8e6de-ff57-42b3-bb42-d54fd3d611c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.270983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8e6de-ff57-42b3-bb42-d54fd3d611c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.271253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8e6de-ff57-42b3-bb42-d54fd3d611c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.271893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8e6de-ff57-42b3-bb42-d54fd3d611c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.273349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545b2ac-6fd7-4fff-8f15-77892dfe2adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.273558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8e6de-ff57-42b3-bb42-d54fd3d611c9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.275012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545b2ac-6fd7-4fff-8f15-77892dfe2adb service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.284518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1159/2290] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:51:06.288236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8e6de-ff57-42b3-bb42-d54fd3d611c9 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.289155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1132/2291] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:06.298806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8e6de-ff57-42b3-bb42-d54fd3d611c9 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.303522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fd172-077f-43c1-8179-62083cde904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.304308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fd172-077f-43c1-8179-62083cde904b admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.304551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fd172-077f-43c1-8179-62083cde904b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.304862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fd172-077f-43c1-8179-62083cde904b admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.305849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fd172-077f-43c1-8179-62083cde904b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.307445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fd172-077f-43c1-8179-62083cde904b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.322312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1160/2292] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:51:06.326128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545b2ac-6fd7-4fff-8f15-77892dfe2adb service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.337649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545b2ac-6fd7-4fff-8f15-77892dfe2adb service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.342861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7bd95-215a-4d0b-b40c-bb2325f0bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.343736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7bd95-215a-4d0b-b40c-bb2325f0bb72 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.344258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7bd95-215a-4d0b-b40c-bb2325f0bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.344729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7bd95-215a-4d0b-b40c-bb2325f0bb72 admin admin] PATH_INFO: `/v3/projects/ea7f5a3f3f29411dbcabbae436db7865/users/e5829f9550be43ad97f8621303a2f28c/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.362208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7bd95-215a-4d0b-b40c-bb2325f0bb72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7f5a3f3f29411dbcabbae436db7865, user_id=e5829f9550be43ad97f8621303a2f28c, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.365190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7bd95-215a-4d0b-b40c-bb2325f0bb72 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.389147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1133/2293] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:51:06 2022] PUT /identity/v3/projects/ea7f5a3f3f29411dbcabbae436db7865/users/e5829f9550be43ad97f8621303a2f28c/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:51:06.397736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120510a-4dac-42f5-adc5-21414020b918 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.398179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120510a-4dac-42f5-adc5-21414020b918 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.398621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120510a-4dac-42f5-adc5-21414020b918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.428104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c120510a-4dac-42f5-adc5-21414020b918 None None] MFA Rules not processed for user `e5829f9550be43ad97f8621303a2f28c`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:06.429430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c120510a-4dac-42f5-adc5-21414020b918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:06.505959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1161/2294] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:06.510321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7bd95-215a-4d0b-b40c-bb2325f0bb72 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.583193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7bd95-215a-4d0b-b40c-bb2325f0bb72 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.588149 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46de07-7d0d-49c6-a633-1b81ea7233aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.588834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46de07-7d0d-49c6-a633-1b81ea7233aa admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.589344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46de07-7d0d-49c6-a633-1b81ea7233aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.589699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46de07-7d0d-49c6-a633-1b81ea7233aa admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.590421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46de07-7d0d-49c6-a633-1b81ea7233aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.592444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46de07-7d0d-49c6-a633-1b81ea7233aa admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.625032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1134/2295] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:51:06 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:51:06.629484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120510a-4dac-42f5-adc5-21414020b918 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.642654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120510a-4dac-42f5-adc5-21414020b918 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.704496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:06.718807 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4295a0-fcf0-404e-a48e-28b3d584aeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.719701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4295a0-fcf0-404e-a48e-28b3d584aeb8 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.719910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4295a0-fcf0-404e-a48e-28b3d584aeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.720005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4295a0-fcf0-404e-a48e-28b3d584aeb8 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.720610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4295a0-fcf0-404e-a48e-28b3d584aeb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.722575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4295a0-fcf0-404e-a48e-28b3d584aeb8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.749359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1162/2296] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:51:06 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:06.751676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46de07-7d0d-49c6-a633-1b81ea7233aa admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.763841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46de07-7d0d-49c6-a633-1b81ea7233aa admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.769567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109fbcc-b297-40dc-8973-92fefa880012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.770283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109fbcc-b297-40dc-8973-92fefa880012 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.770630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109fbcc-b297-40dc-8973-92fefa880012 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.770910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109fbcc-b297-40dc-8973-92fefa880012 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.771585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109fbcc-b297-40dc-8973-92fefa880012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.773259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109fbcc-b297-40dc-8973-92fefa880012 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.786193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1135/2297] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:51:06.790504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4295a0-fcf0-404e-a48e-28b3d584aeb8 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.803711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4295a0-fcf0-404e-a48e-28b3d584aeb8 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.809575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.810410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.810836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.811260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] PATH_INFO: `/v3/projects/40ca7f4e27b747e9af138255a991bbf5/users/bceafc47d7e84271aeb1bb46c0b5d6cc/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.836507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ca7f4e27b747e9af138255a991bbf5, user_id=bceafc47d7e84271aeb1bb46c0b5d6cc, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.839890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:06.867988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1163/2298] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:51:06 2022] PUT /identity/v3/projects/40ca7f4e27b747e9af138255a991bbf5/users/bceafc47d7e84271aeb1bb46c0b5d6cc/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:06.872907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109fbcc-b297-40dc-8973-92fefa880012 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.886210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109fbcc-b297-40dc-8973-92fefa880012 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:06.900761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:06.902822 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:06.991311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de3d9e-067b-4e12-aef6-678a88a24ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:06.991940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de3d9e-067b-4e12-aef6-678a88a24ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:06.992271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de3d9e-067b-4e12-aef6-678a88a24ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:06.993649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de3d9e-067b-4e12-aef6-678a88a24ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:06.994700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de3d9e-067b-4e12-aef6-678a88a24ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:06.997332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de3d9e-067b-4e12-aef6-678a88a24ee1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:07.012519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1136/2299] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:51:07.017826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de3d9e-067b-4e12-aef6-678a88a24ee1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:07.018429 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f9a98-5c64-425e-8333-7add61875911 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:07.024259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:07.024996 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:07.025443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:07.025746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] PATH_INFO: `/v3/projects/f61b5a9aba3d43a9a65f00c475a882cb` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:07.027041 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61b5a9aba3d43a9a65f00c475a882cb)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:07.029186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:07.035897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de3d9e-067b-4e12-aef6-678a88a24ee1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:07.049595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2959c09-806b-47be-8efa-490b9b789b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:07.051143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959c09-806b-47be-8efa-490b9b789b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:07.051452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959c09-806b-47be-8efa-490b9b789b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:07.051807 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959c09-806b-47be-8efa-490b9b789b8d admin admin] PATH_INFO: `/v3/projects/40ca7f4e27b747e9af138255a991bbf5/users/bceafc47d7e84271aeb1bb46c0b5d6cc/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:07.053864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2959c09-806b-47be-8efa-490b9b789b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ca7f4e27b747e9af138255a991bbf5, user_id=bceafc47d7e84271aeb1bb46c0b5d6cc, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:07.056804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2959c09-806b-47be-8efa-490b9b789b8d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:07.070312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1137/2300] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:51:07 2022] PUT /identity/v3/projects/40ca7f4e27b747e9af138255a991bbf5/users/bceafc47d7e84271aeb1bb46c0b5d6cc/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:07.080402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92d948-9294-4715-9dea-9b8fbaf5e0a0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:07.080963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92d948-9294-4715-9dea-9b8fbaf5e0a0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:07.081374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92d948-9294-4715-9dea-9b8fbaf5e0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:07.094910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61b5a9aba3d43a9a65f00c475a882cb', 'request_id': 'req-6b91060d-afcd-48b5-ae51-83891c64ee01'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:07.102991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61b5a9aba3d43a9a65f00c475a882cb', 'request_id': 'req-6b91060d-afcd-48b5-ae51-83891c64ee01'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:07.105357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:07.109405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61b5a9aba3d43a9a65f00c475a882cb', 'request_id': 'req-6b91060d-afcd-48b5-ae51-83891c64ee01'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:07.117930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-de92d948-9294-4715-9dea-9b8fbaf5e0a0 None None] MFA Rules not processed for user `bceafc47d7e84271aeb1bb46c0b5d6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:07.119913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-de92d948-9294-4715-9dea-9b8fbaf5e0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:07.128575 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] The token cache is being invalidate because project f61b5a9aba3d43a9a65f00c475a882cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:51:07.129081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:07.130594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:07.132466 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1164/2301] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:51:06 2022] DELETE /identity/v3/projects/f61b5a9aba3d43a9a65f00c475a882cb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:07.264520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1138/2302] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:51:07.278188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:07.279708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:07.378616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91060d-afcd-48b5-ae51-83891c64ee01 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:07.382361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:07.383059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:07.383337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:07.383606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:07.385455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:07.392285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:07.451680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:07.453163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:07.463469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1165/2303] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:07.954270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bc17c-7b9e-4cdd-9114-06e50c477b20 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:07.954270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bc17c-7b9e-4cdd-9114-06e50c477b20 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:07.954757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73bc17c-7b9e-4cdd-9114-06e50c477b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:07.974326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b73bc17c-7b9e-4cdd-9114-06e50c477b20 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:07.975532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b73bc17c-7b9e-4cdd-9114-06e50c477b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:08.043923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1139/2304] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:51:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:08.094528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:08.096396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:08.181821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85340fe5-77c6-4b5b-affd-47b7c7770aec service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:08.187786 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727bc6c-c182-45a3-97b2-dba8a425b8e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:08.188469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727bc6c-c182-45a3-97b2-dba8a425b8e9 service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:08.188812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727bc6c-c182-45a3-97b2-dba8a425b8e9 service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:08.189375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727bc6c-c182-45a3-97b2-dba8a425b8e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:08.198997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727bc6c-c182-45a3-97b2-dba8a425b8e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:08.200651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727bc6c-c182-45a3-97b2-dba8a425b8e9 service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:08.209961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1166/2305] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:51:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:51:08.784711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170664e6-75c2-41c8-b489-57654af93cfd None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:08.785182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170664e6-75c2-41c8-b489-57654af93cfd None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:08.785599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170664e6-75c2-41c8-b489-57654af93cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:08.817842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-170664e6-75c2-41c8-b489-57654af93cfd None None] MFA Rules not processed for user `6f366d6ae3b4404f9fa4343913f67998`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:08.818406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-170664e6-75c2-41c8-b489-57654af93cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:08.906776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677c631-251a-424c-8af4-0e6d2702a400 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:08.907524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677c631-251a-424c-8af4-0e6d2702a400 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:08.907808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677c631-251a-424c-8af4-0e6d2702a400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:08.923594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1140/2306] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:08.936398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-1677c631-251a-424c-8af4-0e6d2702a400 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:08.938949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-1677c631-251a-424c-8af4-0e6d2702a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:08.957767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1167/2307] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 01:51:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:51:08.978857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170664e6-75c2-41c8-b489-57654af93cfd None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:08.999196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170664e6-75c2-41c8-b489-57654af93cfd None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:09.004795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae676e-94ea-488d-8b6a-5a49a048185a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:09.005472 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae676e-94ea-488d-8b6a-5a49a048185a service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:09.005779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae676e-94ea-488d-8b6a-5a49a048185a service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:09.006049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae676e-94ea-488d-8b6a-5a49a048185a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:09.015709 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae676e-94ea-488d-8b6a-5a49a048185a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:09.018165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae676e-94ea-488d-8b6a-5a49a048185a service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:09.026711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1141/2308] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:51:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:51:09.157147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677c631-251a-424c-8af4-0e6d2702a400 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:09.170014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677c631-251a-424c-8af4-0e6d2702a400 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:09.173985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:09.174730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:09.175101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:09.175470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:09.177447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:09.271821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:09.273412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:09.284636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1168/2309] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:09.373013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39806-0d99-40a2-9d7e-f3314e21f54a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:09.373463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39806-0d99-40a2-9d7e-f3314e21f54a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:09.373836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab39806-0d99-40a2-9d7e-f3314e21f54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:09.396875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-3ab39806-0d99-40a2-9d7e-f3314e21f54a None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:09.398580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3ab39806-0d99-40a2-9d7e-f3314e21f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:09.410182 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1142/2310] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:51:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:09.425310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:09.438706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23860ceb-8a56-4e8d-ab93-f990c61162e3 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:09.444171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66312c3-d5a4-4828-b61c-1c50345194cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:09.444917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66312c3-d5a4-4828-b61c-1c50345194cb service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:09.445229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66312c3-d5a4-4828-b61c-1c50345194cb service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:09.445504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66312c3-d5a4-4828-b61c-1c50345194cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:09.455267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66312c3-d5a4-4828-b61c-1c50345194cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:09.456861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66312c3-d5a4-4828-b61c-1c50345194cb service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:09.466260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1169/2311] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:51:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:51:09.883432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61c0e0-9674-46ad-a5e6-440607684ba9 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:09.883707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61c0e0-9674-46ad-a5e6-440607684ba9 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:09.884094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61c0e0-9674-46ad-a5e6-440607684ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:09.929561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-de61c0e0-9674-46ad-a5e6-440607684ba9 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:09.945277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-de61c0e0-9674-46ad-a5e6-440607684ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:09.953730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1143/2312] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 01:51:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:09.954338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66312c3-d5a4-4828-b61c-1c50345194cb service swift] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:09.965690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66312c3-d5a4-4828-b61c-1c50345194cb service swift] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:09.971928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:09.972775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:09.973273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:09.973732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:09.984448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:09.986662 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:09.990822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61c0e0-9674-46ad-a5e6-440607684ba9 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:09.998204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-de61c0e0-9674-46ad-a5e6-440607684ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:09.999576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1170/2313] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:51:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 01:51:10.021584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61c0e0-9674-46ad-a5e6-440607684ba9 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:10.027109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d77b86-562a-4f3b-b933-b069c6c1d616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:10.027948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77b86-562a-4f3b-b933-b069c6c1d616 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:10.028423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77b86-562a-4f3b-b933-b069c6c1d616 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:10.028791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77b86-562a-4f3b-b933-b069c6c1d616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:10.040165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d77b86-562a-4f3b-b933-b069c6c1d616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:10.042620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d77b86-562a-4f3b-b933-b069c6c1d616 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:10.057546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1144/2314] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:12.196071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:12.197690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:12.203485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:12.279802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edc3a1-e048-4eb3-a2a9-d9b0751e1b1b service swift] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:12.285596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:12.286398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:12.286858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:12.287138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] PATH_INFO: `/v3/users/56ff8a77507c43fba14f0d997cc912ec` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:12.288302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56ff8a77507c43fba14f0d997cc912ec)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:12.290362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:12.341832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56ff8a77507c43fba14f0d997cc912ec'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.342679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56ff8a77507c43fba14f0d997cc912ec'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.355912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56ff8a77507c43fba14f0d997cc912ec'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.364464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56ff8a77507c43fba14f0d997cc912ec'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.365126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56ff8a77507c43fba14f0d997cc912ec'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.378363 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1171/2315] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:51:12 2022] DELETE /identity/v3/users/56ff8a77507c43fba14f0d997cc912ec => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:12.382813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da433fb8-853b-40c6-8b07-8a00329ca37a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:12.383284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da433fb8-853b-40c6-8b07-8a00329ca37a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:12.383417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da433fb8-853b-40c6-8b07-8a00329ca37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:12.396853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:12.403115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-da433fb8-853b-40c6-8b07-8a00329ca37a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:12.404315 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-da433fb8-853b-40c6-8b07-8a00329ca37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:12.464799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1145/2316] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:12.474641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:12.476039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:12.540070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cfe0d-5da0-4c91-ac99-acf62a595050 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:12.549218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e1e9cc-9ea0-46ea-b6cd-58755157295b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:12.550223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1e9cc-9ea0-46ea-b6cd-58755157295b service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:12.550657 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1e9cc-9ea0-46ea-b6cd-58755157295b service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:12.551257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e1e9cc-9ea0-46ea-b6cd-58755157295b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:12.562991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1e9cc-9ea0-46ea-b6cd-58755157295b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:12.564562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e1e9cc-9ea0-46ea-b6cd-58755157295b service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:12.577189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1172/2317] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:12.794853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da433fb8-853b-40c6-8b07-8a00329ca37a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:12.796244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-da433fb8-853b-40c6-8b07-8a00329ca37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:12.812148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da433fb8-853b-40c6-8b07-8a00329ca37a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:12.820628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:12.821886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:12.822234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:12.822575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] PATH_INFO: `/v3/projects/b88c8a0deaaa4aba915ef5e2c32a427b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:12.823979 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b88c8a0deaaa4aba915ef5e2c32a427b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:12.826133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:12.906239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b88c8a0deaaa4aba915ef5e2c32a427b', 'request_id': 'req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.913260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b88c8a0deaaa4aba915ef5e2c32a427b', 'request_id': 'req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.927380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b88c8a0deaaa4aba915ef5e2c32a427b', 'request_id': 'req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.942938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] The token cache is being invalidate because project b88c8a0deaaa4aba915ef5e2c32a427b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:51:12.943227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.944234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-25b1bbd0-183d-4ee7-a6f3-26f510213aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:12.946570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1146/2318] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:51:12 2022] DELETE /identity/v3/projects/b88c8a0deaaa4aba915ef5e2c32a427b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:12.951528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072dfe6-268d-401e-81a9-ee2ce612dd2c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:12.952079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072dfe6-268d-401e-81a9-ee2ce612dd2c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:12.952353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072dfe6-268d-401e-81a9-ee2ce612dd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:12.974365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e072dfe6-268d-401e-81a9-ee2ce612dd2c None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:12.975655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e072dfe6-268d-401e-81a9-ee2ce612dd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:13.001306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:13.061721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1173/2319] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:13.065029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f992d5-51e8-46e9-8b50-547b750efd6f None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.066255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f992d5-51e8-46e9-8b50-547b750efd6f None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.067062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f992d5-51e8-46e9-8b50-547b750efd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.088020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d6f992d5-51e8-46e9-8b50-547b750efd6f None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:13.089436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d6f992d5-51e8-46e9-8b50-547b750efd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:13.121531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1147/2320] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:13.126088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072dfe6-268d-401e-81a9-ee2ce612dd2c None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:13.139351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072dfe6-268d-401e-81a9-ee2ce612dd2c None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:13.150079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:13.150648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.150980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.151446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.152147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:51:13.152452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:13.154257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:13.169461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1174/2321] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:51:13.174310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f992d5-51e8-46e9-8b50-547b750efd6f None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:13.187884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f992d5-51e8-46e9-8b50-547b750efd6f None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:13.203990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259cdf5-20ed-4fa1-b74f-88655e84d3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:13.205049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259cdf5-20ed-4fa1-b74f-88655e84d3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.205049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259cdf5-20ed-4fa1-b74f-88655e84d3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.205322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259cdf5-20ed-4fa1-b74f-88655e84d3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.205822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259cdf5-20ed-4fa1-b74f-88655e84d3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:13.207917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259cdf5-20ed-4fa1-b74f-88655e84d3e5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:13.263639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1148/2322] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:51:13 2022] POST /identity/v3/projects => generated 379 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:13.269268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:13.302634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3790-0d9d-47d4-baa3-505c47643dc5 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:13.403259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff984387-31d6-4f57-b03f-f3c8ca617128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:13.404062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff984387-31d6-4f57-b03f-f3c8ca617128 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.404606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff984387-31d6-4f57-b03f-f3c8ca617128 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.404982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff984387-31d6-4f57-b03f-f3c8ca617128 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.405916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff984387-31d6-4f57-b03f-f3c8ca617128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:13.408665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff984387-31d6-4f57-b03f-f3c8ca617128 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:13.432790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1175/2323] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:51:13 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:13.437177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259cdf5-20ed-4fa1-b74f-88655e84d3e5 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:13.449007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259cdf5-20ed-4fa1-b74f-88655e84d3e5 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:13.454189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9a3d7-6387-4af2-b7c4-742be1763ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:13.454798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9a3d7-6387-4af2-b7c4-742be1763ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.455073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9a3d7-6387-4af2-b7c4-742be1763ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.455341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9a3d7-6387-4af2-b7c4-742be1763ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.455977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9a3d7-6387-4af2-b7c4-742be1763ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:13.457595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9a3d7-6387-4af2-b7c4-742be1763ff1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:13.472930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1149/2324] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:51:13.474243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff984387-31d6-4f57-b03f-f3c8ca617128 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:13.486805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff984387-31d6-4f57-b03f-f3c8ca617128 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:13.495316 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5b9fc-7913-4927-83b5-b4f03ed9018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:13.496488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b9fc-7913-4927-83b5-b4f03ed9018a admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.496915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b9fc-7913-4927-83b5-b4f03ed9018a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.498162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b9fc-7913-4927-83b5-b4f03ed9018a admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.498984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5b9fc-7913-4927-83b5-b4f03ed9018a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:13.502413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5b9fc-7913-4927-83b5-b4f03ed9018a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:13.516505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1176/2325] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:51:13.520638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9a3d7-6387-4af2-b7c4-742be1763ff1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:13.530882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9a3d7-6387-4af2-b7c4-742be1763ff1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:13.536035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:13.537251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.537251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.537480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] PATH_INFO: `/v3/projects/826001ffb29b4d42a5c299154c17b93e/users/fa5465b818f34778a1d51278653488ed/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.555138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=826001ffb29b4d42a5c299154c17b93e, user_id=fa5465b818f34778a1d51278653488ed, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:13.558183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:13.576012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1150/2326] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:51:13 2022] PUT /identity/v3/projects/826001ffb29b4d42a5c299154c17b93e/users/fa5465b818f34778a1d51278653488ed/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:51:13.580944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39c1b-30db-4576-9564-f152a6841e08 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.581485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39c1b-30db-4576-9564-f152a6841e08 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.581759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e39c1b-30db-4576-9564-f152a6841e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.606448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-96e39c1b-30db-4576-9564-f152a6841e08 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:13.607754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-96e39c1b-30db-4576-9564-f152a6841e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:13.675746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1177/2327] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:13.685475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:13.687299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:13.722135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:13.754103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581285fc-ffe3-4524-86dc-6843cb83f861 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:13.758008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad65b4b-98ef-4ef5-ba90-801d3c20bef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:13.758697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65b4b-98ef-4ef5-ba90-801d3c20bef3 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:13.759073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65b4b-98ef-4ef5-ba90-801d3c20bef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:13.759425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad65b4b-98ef-4ef5-ba90-801d3c20bef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:13.771069 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad65b4b-98ef-4ef5-ba90-801d3c20bef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:13.772913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad65b4b-98ef-4ef5-ba90-801d3c20bef3 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:13.794525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1151/2328] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:14.099524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39c1b-30db-4576-9564-f152a6841e08 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:14.100958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-96e39c1b-30db-4576-9564-f152a6841e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:14.115534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e39c1b-30db-4576-9564-f152a6841e08 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:14.121419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:14.122348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:14.122848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:14.123326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] PATH_INFO: `/v3/users/425e9db5848940769958042e579be565` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:14.124762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425e9db5848940769958042e579be565)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:14.127051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:14.177273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:14.218494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425e9db5848940769958042e579be565'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:14.220361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425e9db5848940769958042e579be565'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:14.237531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425e9db5848940769958042e579be565'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:14.251501 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425e9db5848940769958042e579be565'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:14.252606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425e9db5848940769958042e579be565'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:14.270974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1178/2329] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:51:14 2022] DELETE /identity/v3/users/425e9db5848940769958042e579be565 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:14.275627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9ed7a-ae8c-4a1e-a8d5-bb6559413f78 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:14.276155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9ed7a-ae8c-4a1e-a8d5-bb6559413f78 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:14.276647 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9ed7a-ae8c-4a1e-a8d5-bb6559413f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:14.300459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ceb9ed7a-ae8c-4a1e-a8d5-bb6559413f78 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:14.302155 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ceb9ed7a-ae8c-4a1e-a8d5-bb6559413f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:14.389139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1152/2330] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:14.400225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:14.402343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:14.464873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17187aeb-b0a7-4137-9f87-98d6436cd908 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:14.468726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa1274-6172-4ad4-bbb2-ba4a1bf2c622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:14.469373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1274-6172-4ad4-bbb2-ba4a1bf2c622 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:14.469676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1274-6172-4ad4-bbb2-ba4a1bf2c622 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:14.469900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa1274-6172-4ad4-bbb2-ba4a1bf2c622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:14.483120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa1274-6172-4ad4-bbb2-ba4a1bf2c622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:14.485802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa1274-6172-4ad4-bbb2-ba4a1bf2c622 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:14.496596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1179/2331] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:15.344877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9ed7a-ae8c-4a1e-a8d5-bb6559413f78 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:15.346455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ceb9ed7a-ae8c-4a1e-a8d5-bb6559413f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:15.360737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9ed7a-ae8c-4a1e-a8d5-bb6559413f78 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:15.367895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:15.369880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:15.370821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:15.371506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] PATH_INFO: `/v3/projects/d7b1df9340cd470485c850ec89938b87` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:15.374634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7b1df9340cd470485c850ec89938b87)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:15.378161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:15.482082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b1df9340cd470485c850ec89938b87', 'request_id': 'req-47bf6f59-656e-449d-a279-b9ed43259784'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.494696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b1df9340cd470485c850ec89938b87', 'request_id': 'req-47bf6f59-656e-449d-a279-b9ed43259784'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.512497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b1df9340cd470485c850ec89938b87', 'request_id': 'req-47bf6f59-656e-449d-a279-b9ed43259784'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.557944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] The token cache is being invalidate because project d7b1df9340cd470485c850ec89938b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:51:15.558640 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.559632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.564567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1153/2332] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:51:15 2022] DELETE /identity/v3/projects/d7b1df9340cd470485c850ec89938b87 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:15.565735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa1274-6172-4ad4-bbb2-ba4a1bf2c622 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:15.577915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa1274-6172-4ad4-bbb2-ba4a1bf2c622 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:15.675186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:15.676045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:15.676370 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:15.676604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] PATH_INFO: `/v3/users/e5829f9550be43ad97f8621303a2f28c` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:15.677831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5829f9550be43ad97f8621303a2f28c)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:15.679842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:15.742787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5829f9550be43ad97f8621303a2f28c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.743793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5829f9550be43ad97f8621303a2f28c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.745496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:15.761239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5829f9550be43ad97f8621303a2f28c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.771803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5829f9550be43ad97f8621303a2f28c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.772511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5829f9550be43ad97f8621303a2f28c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:15.788506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1180/2333] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:51:15 2022] DELETE /identity/v3/users/e5829f9550be43ad97f8621303a2f28c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:51:16.861342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:16.863714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:16.885383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf6f59-656e-449d-a279-b9ed43259784 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:16.999417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:17.065028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:17.068890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:17.070580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:17.071013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] PATH_INFO: `/v3/projects/ea7f5a3f3f29411dbcabbae436db7865` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:17.076479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7f5a3f3f29411dbcabbae436db7865)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:17.079568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:17.187541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7f5a3f3f29411dbcabbae436db7865', 'request_id': 'req-80bc1c2f-d768-4991-aa16-5819b1bd0812'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.202263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7f5a3f3f29411dbcabbae436db7865', 'request_id': 'req-80bc1c2f-d768-4991-aa16-5819b1bd0812'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.214621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7f5a3f3f29411dbcabbae436db7865', 'request_id': 'req-80bc1c2f-d768-4991-aa16-5819b1bd0812'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.255694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] The token cache is being invalidate because project ea7f5a3f3f29411dbcabbae436db7865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:51:17.255996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.256661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.261213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1154/2334] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:51:16 2022] DELETE /identity/v3/projects/ea7f5a3f3f29411dbcabbae436db7865 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:17.262775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:17.264312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:17.288421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e82b2c-39de-4a4c-a408-cb5f1b3a1fee admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:17.421783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:17.422437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:17.422721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:17.422997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] PATH_INFO: `/v3/users/bceafc47d7e84271aeb1bb46c0b5d6cc` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:17.425754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bceafc47d7e84271aeb1bb46c0b5d6cc)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:17.427686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:17.505326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bceafc47d7e84271aeb1bb46c0b5d6cc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.506664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bceafc47d7e84271aeb1bb46c0b5d6cc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.526504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bceafc47d7e84271aeb1bb46c0b5d6cc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.538271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bceafc47d7e84271aeb1bb46c0b5d6cc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.539764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e605043-2552-41a1-a276-fd75c81d2ed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bceafc47d7e84271aeb1bb46c0b5d6cc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:17.561379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1181/2335] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:51:17 2022] DELETE /identity/v3/users/bceafc47d7e84271aeb1bb46c0b5d6cc => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:18.095220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.097336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:18.113332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc1c2f-d768-4991-aa16-5819b1bd0812 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:18.195663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:18.196691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.197091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.197426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] PATH_INFO: `/v3/projects/40ca7f4e27b747e9af138255a991bbf5` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.199493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ca7f4e27b747e9af138255a991bbf5)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:18.202436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:18.264532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ca7f4e27b747e9af138255a991bbf5', 'request_id': 'req-eda1e65f-5370-44ed-a521-6b778c082514'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:18.271239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ca7f4e27b747e9af138255a991bbf5', 'request_id': 'req-eda1e65f-5370-44ed-a521-6b778c082514'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:18.278197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ca7f4e27b747e9af138255a991bbf5', 'request_id': 'req-eda1e65f-5370-44ed-a521-6b778c082514'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:18.293535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] The token cache is being invalidate because project 40ca7f4e27b747e9af138255a991bbf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:51:18.293927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:18.294574 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-eda1e65f-5370-44ed-a521-6b778c082514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:51:18.296084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1155/2336] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:51:18 2022] DELETE /identity/v3/projects/40ca7f4e27b747e9af138255a991bbf5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:18.300965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d036b5-4965-4b25-b988-ff2ecdb3af21 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.301478 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d036b5-4965-4b25-b988-ff2ecdb3af21 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.301589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d036b5-4965-4b25-b988-ff2ecdb3af21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.326499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-02d036b5-4965-4b25-b988-ff2ecdb3af21 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:18.327740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-02d036b5-4965-4b25-b988-ff2ecdb3af21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:18.386735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:18.425660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1182/2337] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:18.427620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61035b63-10a6-4fbd-8559-a670a270d089 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.427910 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61035b63-10a6-4fbd-8559-a670a270d089 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.428186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61035b63-10a6-4fbd-8559-a670a270d089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.459879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-61035b63-10a6-4fbd-8559-a670a270d089 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:18.461447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-61035b63-10a6-4fbd-8559-a670a270d089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:18.474855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1156/2338] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:18.477526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d036b5-4965-4b25-b988-ff2ecdb3af21 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.495862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d036b5-4965-4b25-b988-ff2ecdb3af21 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:18.501922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:18.502912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.503258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.503598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.504323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:51:18.504637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:18.509857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:18.531692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1183/2339] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:51:18.535674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61035b63-10a6-4fbd-8559-a670a270d089 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.548284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61035b63-10a6-4fbd-8559-a670a270d089 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:18.554426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594c22f-ec3c-408f-8db3-ba2c5ee46297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:18.555058 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594c22f-ec3c-408f-8db3-ba2c5ee46297 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.555334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594c22f-ec3c-408f-8db3-ba2c5ee46297 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.555604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594c22f-ec3c-408f-8db3-ba2c5ee46297 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.556244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594c22f-ec3c-408f-8db3-ba2c5ee46297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:18.559029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594c22f-ec3c-408f-8db3-ba2c5ee46297 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:18.568228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:18.603161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1157/2340] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:51:18 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:18.607362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.619663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc4c71-4a59-47d7-ab68-30d9f7435567 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:18.650606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340a4c-d049-49ed-82e5-0f7af622f968 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.650930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340a4c-d049-49ed-82e5-0f7af622f968 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.651166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce340a4c-d049-49ed-82e5-0f7af622f968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.686036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ce340a4c-d049-49ed-82e5-0f7af622f968 None None] MFA Rules not processed for user `fa5465b818f34778a1d51278653488ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:18.687895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ce340a4c-d049-49ed-82e5-0f7af622f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:18.698158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8206e4-94ed-4c87-85f0-82b26cabb011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:18.698987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8206e4-94ed-4c87-85f0-82b26cabb011 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.699262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8206e4-94ed-4c87-85f0-82b26cabb011 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.699529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8206e4-94ed-4c87-85f0-82b26cabb011 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.700364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8206e4-94ed-4c87-85f0-82b26cabb011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:18.703338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8206e4-94ed-4c87-85f0-82b26cabb011 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:18.735121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1184/2341] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:51:18 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:18.738899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8206e4-94ed-4c87-85f0-82b26cabb011 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.749445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8206e4-94ed-4c87-85f0-82b26cabb011 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:18.759083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a2f37-1a34-4e43-b50b-44202e2b803a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:18.759712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a2f37-1a34-4e43-b50b-44202e2b803a admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.759996 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a2f37-1a34-4e43-b50b-44202e2b803a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.760269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a2f37-1a34-4e43-b50b-44202e2b803a admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.760919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a2f37-1a34-4e43-b50b-44202e2b803a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:18.762564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a2f37-1a34-4e43-b50b-44202e2b803a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:18.773710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1185/2342] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:51:18.777593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a2f37-1a34-4e43-b50b-44202e2b803a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.787679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a2f37-1a34-4e43-b50b-44202e2b803a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:18.801724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1158/2343] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:51:18.806237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce340a4c-d049-49ed-82e5-0f7af622f968 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.808992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ce340a4c-d049-49ed-82e5-0f7af622f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:18.811234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45215-7289-4597-bd4f-ce13b7c69fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:18.813512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45215-7289-4597-bd4f-ce13b7c69fea admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.813512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45215-7289-4597-bd4f-ce13b7c69fea admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.813802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae45215-7289-4597-bd4f-ce13b7c69fea admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.815376 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae45215-7289-4597-bd4f-ce13b7c69fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:18.816125 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae45215-7289-4597-bd4f-ce13b7c69fea admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:18.831575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce340a4c-d049-49ed-82e5-0f7af622f968 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:18.834960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1186/2344] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:51:18.835502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45215-7289-4597-bd4f-ce13b7c69fea admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.836874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf648824-f165-450c-b501-a63ce8ba18aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:18.837596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648824-f165-450c-b501-a63ce8ba18aa admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.837946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648824-f165-450c-b501-a63ce8ba18aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.838289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648824-f165-450c-b501-a63ce8ba18aa admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.839286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf648824-f165-450c-b501-a63ce8ba18aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:18.841450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf648824-f165-450c-b501-a63ce8ba18aa admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:18.845852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae45215-7289-4597-bd4f-ce13b7c69fea admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:18.855229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aab2b2-b3d8-48b6-b044-2521b91f47c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:18.855910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aab2b2-b3d8-48b6-b044-2521b91f47c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.856249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aab2b2-b3d8-48b6-b044-2521b91f47c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.856519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aab2b2-b3d8-48b6-b044-2521b91f47c8 admin admin] PATH_INFO: `/v3/projects/2b2689737cbe48f1a905a987291a7203/users/5f9d00f4c4f04f3f855405414c671391/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.883259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aab2b2-b3d8-48b6-b044-2521b91f47c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b2689737cbe48f1a905a987291a7203, user_id=5f9d00f4c4f04f3f855405414c671391, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:18.884978 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1159/2345] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:51:18 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:18.886213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aab2b2-b3d8-48b6-b044-2521b91f47c8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:18.888784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf648824-f165-450c-b501-a63ce8ba18aa admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:18.905557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1187/2346] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:51:18 2022] PUT /identity/v3/projects/2b2689737cbe48f1a905a987291a7203/users/5f9d00f4c4f04f3f855405414c671391/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:18.918834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fd15-4217-4a6a-ae2f-042e4b4b1590 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:18.918834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fd15-4217-4a6a-ae2f-042e4b4b1590 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:18.918834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fd15-4217-4a6a-ae2f-042e4b4b1590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:18.962805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5cb6fd15-4217-4a6a-ae2f-042e4b4b1590 None None] MFA Rules not processed for user `5f9d00f4c4f04f3f855405414c671391`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:18.964989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5cb6fd15-4217-4a6a-ae2f-042e4b4b1590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:18.981724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf648824-f165-450c-b501-a63ce8ba18aa admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:19.057663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1188/2347] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:51:19.067162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c537506-4d44-488b-8989-dea3c376850e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:19.068325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c537506-4d44-488b-8989-dea3c376850e admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:19.068682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c537506-4d44-488b-8989-dea3c376850e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:19.068971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c537506-4d44-488b-8989-dea3c376850e admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:19.069702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c537506-4d44-488b-8989-dea3c376850e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:19.070004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6fd15-4217-4a6a-ae2f-042e4b4b1590 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:19.071396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5cb6fd15-4217-4a6a-ae2f-042e4b4b1590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:19.074546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c537506-4d44-488b-8989-dea3c376850e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:19.078016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:19.086214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:19.107561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1160/2348] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:51:18 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:51:19.112352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c537506-4d44-488b-8989-dea3c376850e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:19.122822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c537506-4d44-488b-8989-dea3c376850e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:19.127736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48c033-1f08-450f-8363-879f9b7bd9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:19.128343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48c033-1f08-450f-8363-879f9b7bd9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:19.128619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48c033-1f08-450f-8363-879f9b7bd9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:19.128890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48c033-1f08-450f-8363-879f9b7bd9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:19.129562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48c033-1f08-450f-8363-879f9b7bd9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:19.131461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48c033-1f08-450f-8363-879f9b7bd9a1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:19.142617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6fd15-4217-4a6a-ae2f-042e4b4b1590 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:19.143491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1161/2349] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:51:19.146946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e96af-0fc4-4ce6-b15a-7ec5b3b8d15a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:19.147209 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48c033-1f08-450f-8363-879f9b7bd9a1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:19.147816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e96af-0fc4-4ce6-b15a-7ec5b3b8d15a service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:19.148101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e96af-0fc4-4ce6-b15a-7ec5b3b8d15a service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:19.148371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e96af-0fc4-4ce6-b15a-7ec5b3b8d15a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:19.158341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e96af-0fc4-4ce6-b15a-7ec5b3b8d15a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:19.159259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48c033-1f08-450f-8363-879f9b7bd9a1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:19.160468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e96af-0fc4-4ce6-b15a-7ec5b3b8d15a service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:19.164253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb89ff3-0804-40a8-8b83-3638bcf477fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:19.165422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89ff3-0804-40a8-8b83-3638bcf477fa admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:19.165798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89ff3-0804-40a8-8b83-3638bcf477fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:19.166159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89ff3-0804-40a8-8b83-3638bcf477fa admin admin] PATH_INFO: `/v3/projects/bd10498303de47c69683c94f24b83f18/users/4cf02609cda94190832e124ac3dbd043/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:19.173996 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1189/2350] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:51:19.184545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb89ff3-0804-40a8-8b83-3638bcf477fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd10498303de47c69683c94f24b83f18, user_id=4cf02609cda94190832e124ac3dbd043, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:19.187477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb89ff3-0804-40a8-8b83-3638bcf477fa admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:19.202418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1162/2351] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:51:19 2022] PUT /identity/v3/projects/bd10498303de47c69683c94f24b83f18/users/4cf02609cda94190832e124ac3dbd043/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:51:19.206460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e96af-0fc4-4ce6-b15a-7ec5b3b8d15a service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:19.220420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e96af-0fc4-4ce6-b15a-7ec5b3b8d15a service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:19.286255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731be08c-605f-4fcb-9340-abdcf672431b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:19.286888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731be08c-605f-4fcb-9340-abdcf672431b admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:19.287200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731be08c-605f-4fcb-9340-abdcf672431b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:19.287435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731be08c-605f-4fcb-9340-abdcf672431b admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:19.288083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731be08c-605f-4fcb-9340-abdcf672431b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:19.289881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731be08c-605f-4fcb-9340-abdcf672431b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:19.300980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1190/2352] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:51:19.305310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb89ff3-0804-40a8-8b83-3638bcf477fa admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:19.322777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb89ff3-0804-40a8-8b83-3638bcf477fa admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:19.348760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:19.351701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:19.352088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:19.352492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] PATH_INFO: `/v3/projects/bd10498303de47c69683c94f24b83f18/users/4cf02609cda94190832e124ac3dbd043/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:19.355879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd10498303de47c69683c94f24b83f18, user_id=4cf02609cda94190832e124ac3dbd043, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:19.362642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:19.377369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1163/2353] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:51:19 2022] PUT /identity/v3/projects/bd10498303de47c69683c94f24b83f18/users/4cf02609cda94190832e124ac3dbd043/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:51:25.073489 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105c423-88e4-4a40-b978-18a435759cbb None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:25.073963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105c423-88e4-4a40-b978-18a435759cbb None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:25.077141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105c423-88e4-4a40-b978-18a435759cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:25.129607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2105c423-88e4-4a40-b978-18a435759cbb None None] MFA Rules not processed for user `4cf02609cda94190832e124ac3dbd043`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:25.134348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2105c423-88e4-4a40-b978-18a435759cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:25.279132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1191/2354] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:25.294931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:25.298052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:25.464515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752f7c4-3934-46c8-8148-555d2bd1b430 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:25.478284 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888cfaa-3225-44ae-bb3b-b52e5bf354ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:25.479266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888cfaa-3225-44ae-bb3b-b52e5bf354ac service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:25.479823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888cfaa-3225-44ae-bb3b-b52e5bf354ac service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:25.480357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888cfaa-3225-44ae-bb3b-b52e5bf354ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:25.596734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888cfaa-3225-44ae-bb3b-b52e5bf354ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:25.599041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888cfaa-3225-44ae-bb3b-b52e5bf354ac service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:25.628388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1164/2355] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:29.382307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cf8ff-f768-45ca-a8ea-ebb86d00cde2 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:29.382881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cf8ff-f768-45ca-a8ea-ebb86d00cde2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:29.382881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cf8ff-f768-45ca-a8ea-ebb86d00cde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:29.403313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2f9cf8ff-f768-45ca-a8ea-ebb86d00cde2 None None] MFA Rules not processed for user `3153c9ff011f4ab8958fc49795e6f8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:29.404736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2f9cf8ff-f768-45ca-a8ea-ebb86d00cde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:29.431415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:29.468231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1192/2356] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:51:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:35.609327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:INSERT=5,keystone:SELECT=78 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:37.468715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac961-65b1-4ecb-9b7b-7f0e30bc76bb None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:37.469763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac961-65b1-4ecb-9b7b-7f0e30bc76bb None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:37.470305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac961-65b1-4ecb-9b7b-7f0e30bc76bb None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:37.472199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1165/2357] 10.222.0.15 () {54 vars in 961 bytes} [Fri Sep 9 01:51:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:51:37.478199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcb130-cf98-4c2c-94a8-0b5ce26a80f0 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:37.479155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcb130-cf98-4c2c-94a8-0b5ce26a80f0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:37.479506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcb130-cf98-4c2c-94a8-0b5ce26a80f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:37.503984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6cdcb130-cf98-4c2c-94a8-0b5ce26a80f0 None None] MFA Rules not processed for user `365e1f44e9d849118e64975b4c3d674b`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:37.505497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6cdcb130-cf98-4c2c-94a8-0b5ce26a80f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:37.602012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1193/2358] 10.222.0.15 () {58 vars in 1058 bytes} [Fri Sep 9 01:51:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:37.610914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ac961-65b1-4ecb-9b7b-7f0e30bc76bb None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:37.634424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ac961-65b1-4ecb-9b7b-7f0e30bc76bb None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:37.647356 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:37.647911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:37.648188 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:37.648589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:37.650665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:37.666445 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:37.668046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:37.681678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1166/2359] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:51:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:51:47.590596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=41 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:47.675810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=6 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:51:56.945255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106efab1-6084-4408-b4d7-43dec1c8ad46 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:56.945739 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106efab1-6084-4408-b4d7-43dec1c8ad46 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:56.945845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106efab1-6084-4408-b4d7-43dec1c8ad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:56.966890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-106efab1-6084-4408-b4d7-43dec1c8ad46 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:51:56.968510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-106efab1-6084-4408-b4d7-43dec1c8ad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:57.036950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1194/2360] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:51:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:51:57.050422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:51:57.051844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:51:57.119495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f2d8e-c094-4f68-9c23-003e6e913d18 service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:51:57.125258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069ce81-031c-4afe-b264-4edc7ae69728 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:51:57.125853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069ce81-031c-4afe-b264-4edc7ae69728 service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:51:57.126126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069ce81-031c-4afe-b264-4edc7ae69728 service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:51:57.126462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069ce81-031c-4afe-b264-4edc7ae69728 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:51:57.134109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069ce81-031c-4afe-b264-4edc7ae69728 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:51:57.135665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3069ce81-031c-4afe-b264-4edc7ae69728 service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:51:57.143800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1167/2361] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:51:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:52:07.029305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=25 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:07.140269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=32 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:15.127187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d0cb0-d8e3-4a2e-b34f-cc19ab6c2b08 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:15.127782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d0cb0-d8e3-4a2e-b34f-cc19ab6c2b08 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:15.127782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d0cb0-d8e3-4a2e-b34f-cc19ab6c2b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:15.161313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6f6d0cb0-d8e3-4a2e-b34f-cc19ab6c2b08 None None] MFA Rules not processed for user `fa5465b818f34778a1d51278653488ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:15.162962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6f6d0cb0-d8e3-4a2e-b34f-cc19ab6c2b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:15.188037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1195/2362] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:17.920890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713503-1f9b-4b8c-9530-89830f52a31b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:17.921420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713503-1f9b-4b8c-9530-89830f52a31b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:17.921638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f713503-1f9b-4b8c-9530-89830f52a31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:17.943459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-5f713503-1f9b-4b8c-9530-89830f52a31b None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:17.944912 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5f713503-1f9b-4b8c-9530-89830f52a31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:17.955637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1168/2363] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 01:52:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:52:17.968095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d0cb0-d8e3-4a2e-b34f-cc19ab6c2b08 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:17.980801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d0cb0-d8e3-4a2e-b34f-cc19ab6c2b08 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:17.987839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952566c-03aa-4f66-bc1d-dd01a1d4435d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:17.987969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952566c-03aa-4f66-bc1d-dd01a1d4435d service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:17.988208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952566c-03aa-4f66-bc1d-dd01a1d4435d service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:17.988477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952566c-03aa-4f66-bc1d-dd01a1d4435d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:17.997938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952566c-03aa-4f66-bc1d-dd01a1d4435d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:17.999559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952566c-03aa-4f66-bc1d-dd01a1d4435d service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:18.007686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1196/2364] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:52:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:52:20.137221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713503-1f9b-4b8c-9530-89830f52a31b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:20.139442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5f713503-1f9b-4b8c-9530-89830f52a31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:20.218899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f713503-1f9b-4b8c-9530-89830f52a31b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:20.224329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:20.225013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:20.225322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:20.225593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] PATH_INFO: `/v3/users/6f366d6ae3b4404f9fa4343913f67998` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:20.226712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f366d6ae3b4404f9fa4343913f67998)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:20.228690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:20.278039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f366d6ae3b4404f9fa4343913f67998'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.278695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f366d6ae3b4404f9fa4343913f67998'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.286498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f366d6ae3b4404f9fa4343913f67998'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.298214 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f366d6ae3b4404f9fa4343913f67998'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.304865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3220727-511a-44d1-a662-f09f5f3e46c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f366d6ae3b4404f9fa4343913f67998'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.310586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1169/2365] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:52:20 2022] DELETE /identity/v3/users/6f366d6ae3b4404f9fa4343913f67998 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:20.482164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952566c-03aa-4f66-bc1d-dd01a1d4435d service swift] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:20.494172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952566c-03aa-4f66-bc1d-dd01a1d4435d service swift] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:20.551615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:20.552247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:20.552528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:20.552800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] PATH_INFO: `/v3/projects/f4e0c857a31147f8afe25e68cc3e9f24` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:20.553947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e0c857a31147f8afe25e68cc3e9f24)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:20.555833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:20.601520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e0c857a31147f8afe25e68cc3e9f24', 'request_id': 'req-e56f8422-6df5-4b86-b151-ee4b0e479422'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.607644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e0c857a31147f8afe25e68cc3e9f24', 'request_id': 'req-e56f8422-6df5-4b86-b151-ee4b0e479422'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.613206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e0c857a31147f8afe25e68cc3e9f24', 'request_id': 'req-e56f8422-6df5-4b86-b151-ee4b0e479422'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.626051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] The token cache is being invalidate because project f4e0c857a31147f8afe25e68cc3e9f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:52:20.626749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.627150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e56f8422-6df5-4b86-b151-ee4b0e479422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:20.628427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1197/2366] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:52:20 2022] DELETE /identity/v3/projects/f4e0c857a31147f8afe25e68cc3e9f24 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:20.633215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad1f52-192c-49e3-a35c-79ed3bf52e68 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:20.633498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad1f52-192c-49e3-a35c-79ed3bf52e68 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:20.633769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad1f52-192c-49e3-a35c-79ed3bf52e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:20.653843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-bbad1f52-192c-49e3-a35c-79ed3bf52e68 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:20.655080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bbad1f52-192c-49e3-a35c-79ed3bf52e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:20.716475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1170/2367] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:20.724551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d648-1ebe-4593-bbe7-d881407791f9 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:20.725462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d648-1ebe-4593-bbe7-d881407791f9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:20.725773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9d648-1ebe-4593-bbe7-d881407791f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:20.755202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-0ad9d648-1ebe-4593-bbe7-d881407791f9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:20.756509 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0ad9d648-1ebe-4593-bbe7-d881407791f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:20.766310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1198/2368] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:20.770671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad1f52-192c-49e3-a35c-79ed3bf52e68 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:20.780602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad1f52-192c-49e3-a35c-79ed3bf52e68 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:20.785614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:20.786490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:20.786961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:20.787237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:20.788083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:52:20.788375 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:20.790166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:20.799489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:20.807421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1171/2369] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:52:20.809716 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9d648-1ebe-4593-bbe7-d881407791f9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:20.820486 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9d648-1ebe-4593-bbe7-d881407791f9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:20.825445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b453b39-3415-4bd6-bb32-85d792afc6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:20.826045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b453b39-3415-4bd6-bb32-85d792afc6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:20.826320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b453b39-3415-4bd6-bb32-85d792afc6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:20.826653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b453b39-3415-4bd6-bb32-85d792afc6d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:20.827280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b453b39-3415-4bd6-bb32-85d792afc6d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:20.829194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b453b39-3415-4bd6-bb32-85d792afc6d7 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:20.864233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1199/2370] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:52:20 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:52:20.868134 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:20.878811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b321c3-c640-484f-83f7-724792e8acc0 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:20.934102 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6352ae-1e71-466e-b4b3-d64cf894b1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:20.934696 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6352ae-1e71-466e-b4b3-d64cf894b1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:20.934970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6352ae-1e71-466e-b4b3-d64cf894b1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:20.935238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6352ae-1e71-466e-b4b3-d64cf894b1d6 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:20.936483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6352ae-1e71-466e-b4b3-d64cf894b1d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:20.938415 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6352ae-1e71-466e-b4b3-d64cf894b1d6 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:20.960111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1172/2371] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:52:20 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:52:20.963841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b453b39-3415-4bd6-bb32-85d792afc6d7 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:20.974285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b453b39-3415-4bd6-bb32-85d792afc6d7 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:20.978929 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4f7a-0266-49ae-856f-db105e9163d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:20.979548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4f7a-0266-49ae-856f-db105e9163d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:20.979825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4f7a-0266-49ae-856f-db105e9163d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:20.980096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a4f7a-0266-49ae-856f-db105e9163d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:20.980729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a4f7a-0266-49ae-856f-db105e9163d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:20.982332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a4f7a-0266-49ae-856f-db105e9163d4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:20.997076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1200/2372] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:52:21.001090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6352ae-1e71-466e-b4b3-d64cf894b1d6 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:21.014553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6352ae-1e71-466e-b4b3-d64cf894b1d6 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:21.019433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe002829-044b-4ca6-bbbc-adb6c1606d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:21.020033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002829-044b-4ca6-bbbc-adb6c1606d7e admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:21.020308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002829-044b-4ca6-bbbc-adb6c1606d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:21.023975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe002829-044b-4ca6-bbbc-adb6c1606d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:21.023975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe002829-044b-4ca6-bbbc-adb6c1606d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:21.023975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe002829-044b-4ca6-bbbc-adb6c1606d7e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:21.035195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1173/2373] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:52:21.039502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4f7a-0266-49ae-856f-db105e9163d4 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:21.050189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a4f7a-0266-49ae-856f-db105e9163d4 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:21.055452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:21.055754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:21.056028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:21.056297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] PATH_INFO: `/v3/projects/affdf63b83a4457da416c9442418e6e4/users/656b76745b374947ba6cac54c70fd9bc/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:21.073145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=affdf63b83a4457da416c9442418e6e4, user_id=656b76745b374947ba6cac54c70fd9bc, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:21.075833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:21.090199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1201/2374] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:52:21 2022] PUT /identity/v3/projects/affdf63b83a4457da416c9442418e6e4/users/656b76745b374947ba6cac54c70fd9bc/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:52:21.100184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb34fc-d470-4e1e-b0b1-bf1d77eea481 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:21.100468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb34fc-d470-4e1e-b0b1-bf1d77eea481 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:21.100738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb34fc-d470-4e1e-b0b1-bf1d77eea481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:21.130782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7ffb34fc-d470-4e1e-b0b1-bf1d77eea481 None None] MFA Rules not processed for user `656b76745b374947ba6cac54c70fd9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:21.132054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7ffb34fc-d470-4e1e-b0b1-bf1d77eea481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:21.204543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1174/2375] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:21.215475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:21.218364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:21.229845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:21.296709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f65d51-5328-4115-8567-398b696b3256 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:21.301761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a0f6b-c8a2-466d-b604-0f0dc22cdacf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:21.302380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0f6b-c8a2-466d-b604-0f0dc22cdacf service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:21.302717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0f6b-c8a2-466d-b604-0f0dc22cdacf service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:21.303136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a0f6b-c8a2-466d-b604-0f0dc22cdacf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:21.312769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a0f6b-c8a2-466d-b604-0f0dc22cdacf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:21.314369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a0f6b-c8a2-466d-b604-0f0dc22cdacf service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:21.323648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1202/2376] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:52:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:52:22.167225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb34fc-d470-4e1e-b0b1-bf1d77eea481 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:22.168753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7ffb34fc-d470-4e1e-b0b1-bf1d77eea481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:22.235342 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb34fc-d470-4e1e-b0b1-bf1d77eea481 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:22.240557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:22.241233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:22.241516 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:22.241847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] PATH_INFO: `/v3/users/5f9d00f4c4f04f3f855405414c671391` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:22.243159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f9d00f4c4f04f3f855405414c671391)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:22.245163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:22.259389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:22.302142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f9d00f4c4f04f3f855405414c671391'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.303123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f9d00f4c4f04f3f855405414c671391'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.310400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f9d00f4c4f04f3f855405414c671391'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.323157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f9d00f4c4f04f3f855405414c671391'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.330749 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f9d00f4c4f04f3f855405414c671391'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.336825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1175/2377] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:52:22 2022] DELETE /identity/v3/users/5f9d00f4c4f04f3f855405414c671391 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:22.341350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458363-e18a-4b2d-9a70-7720bb239d69 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:22.342147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458363-e18a-4b2d-9a70-7720bb239d69 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:22.342492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75458363-e18a-4b2d-9a70-7720bb239d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:22.363035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-75458363-e18a-4b2d-9a70-7720bb239d69 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:22.364250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-75458363-e18a-4b2d-9a70-7720bb239d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:22.425174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1203/2378] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:22.434689 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:22.436736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:22.493086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2419ea3-24fd-4fe7-bcfa-1e477f3fd70c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:22.497033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f472355-ea5f-45f8-ab36-21c268709c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:22.497656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472355-ea5f-45f8-ab36-21c268709c2f service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:22.497958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472355-ea5f-45f8-ab36-21c268709c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:22.498796 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f472355-ea5f-45f8-ab36-21c268709c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:22.507941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f472355-ea5f-45f8-ab36-21c268709c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:22.509452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f472355-ea5f-45f8-ab36-21c268709c2f service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:22.518873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1176/2379] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:52:22.799350 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75458363-e18a-4b2d-9a70-7720bb239d69 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:22.801146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-75458363-e18a-4b2d-9a70-7720bb239d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:22.824083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75458363-e18a-4b2d-9a70-7720bb239d69 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:22.828933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:22.829881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:22.830331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:22.830612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] PATH_INFO: `/v3/projects/2b2689737cbe48f1a905a987291a7203` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:22.831782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b2689737cbe48f1a905a987291a7203)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:22.833741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:22.892602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b2689737cbe48f1a905a987291a7203', 'request_id': 'req-0432e9fb-ec97-4a91-9be4-2346bed5540c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.899684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b2689737cbe48f1a905a987291a7203', 'request_id': 'req-0432e9fb-ec97-4a91-9be4-2346bed5540c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.907265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b2689737cbe48f1a905a987291a7203', 'request_id': 'req-0432e9fb-ec97-4a91-9be4-2346bed5540c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.926327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] The token cache is being invalidate because project 2b2689737cbe48f1a905a987291a7203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:52:22.926620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.927281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0432e9fb-ec97-4a91-9be4-2346bed5540c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:22.928778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1204/2380] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:52:22 2022] DELETE /identity/v3/projects/2b2689737cbe48f1a905a987291a7203 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:22.935324 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283d326-bf02-4a48-9d10-6d14262c959d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:22.936213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283d326-bf02-4a48-9d10-6d14262c959d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:22.936973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283d326-bf02-4a48-9d10-6d14262c959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:22.958702 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f283d326-bf02-4a48-9d10-6d14262c959d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:22.959914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f283d326-bf02-4a48-9d10-6d14262c959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:23.036493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1177/2381] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:52:23.043167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dc129-47ec-4eba-8430-bb23e6a4f2d8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.044050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dc129-47ec-4eba-8430-bb23e6a4f2d8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.044690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dc129-47ec-4eba-8430-bb23e6a4f2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.069265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d95dc129-47ec-4eba-8430-bb23e6a4f2d8 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:23.070474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d95dc129-47ec-4eba-8430-bb23e6a4f2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:23.081129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1205/2382] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:23.086377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283d326-bf02-4a48-9d10-6d14262c959d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:23.096551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283d326-bf02-4a48-9d10-6d14262c959d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:23.102611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:23.103394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.103680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.103964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.104750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:52:23.105052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:23.106816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:23.120226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1178/2383] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:52:23.124887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dc129-47ec-4eba-8430-bb23e6a4f2d8 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:23.135526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dc129-47ec-4eba-8430-bb23e6a4f2d8 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:23.158559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3d8ea-f52c-4037-9cc4-d78e07ce8079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:23.159292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3d8ea-f52c-4037-9cc4-d78e07ce8079 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.159675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3d8ea-f52c-4037-9cc4-d78e07ce8079 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.160343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3d8ea-f52c-4037-9cc4-d78e07ce8079 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.160801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3d8ea-f52c-4037-9cc4-d78e07ce8079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:23.163020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3d8ea-f52c-4037-9cc4-d78e07ce8079 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:23.208307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1206/2384] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:52:23 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:52:23.212552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:23.224779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60984a76-f6be-4280-bb27-0dd9c8d337fe admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:23.238453 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:23.287864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c156e-f83a-46b7-a9ad-11027f224b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:23.288639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c156e-f83a-46b7-a9ad-11027f224b81 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.288931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c156e-f83a-46b7-a9ad-11027f224b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.289279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c156e-f83a-46b7-a9ad-11027f224b81 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.289900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c156e-f83a-46b7-a9ad-11027f224b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:23.291944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c156e-f83a-46b7-a9ad-11027f224b81 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:23.325443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1179/2385] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:52:23 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:52:23.329441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3d8ea-f52c-4037-9cc4-d78e07ce8079 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:23.336010 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:23.340957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3d8ea-f52c-4037-9cc4-d78e07ce8079 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:23.347139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adc464-1fd7-4061-99b5-ee5e59b91f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:23.347912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adc464-1fd7-4061-99b5-ee5e59b91f78 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.348374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adc464-1fd7-4061-99b5-ee5e59b91f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.348686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adc464-1fd7-4061-99b5-ee5e59b91f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.349560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adc464-1fd7-4061-99b5-ee5e59b91f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:23.351219 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adc464-1fd7-4061-99b5-ee5e59b91f78 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:23.361724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1207/2386] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:52:23.365711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c156e-f83a-46b7-a9ad-11027f224b81 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:23.376928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c156e-f83a-46b7-a9ad-11027f224b81 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:23.381481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffdd67-6fe3-4a5a-a7eb-4a33a77fc2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:23.382068 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffdd67-6fe3-4a5a-a7eb-4a33a77fc2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.382347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffdd67-6fe3-4a5a-a7eb-4a33a77fc2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.382617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffdd67-6fe3-4a5a-a7eb-4a33a77fc2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.383252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffdd67-6fe3-4a5a-a7eb-4a33a77fc2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:23.384822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffdd67-6fe3-4a5a-a7eb-4a33a77fc2f2 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:23.395277 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1180/2387] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:52:23.399311 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adc464-1fd7-4061-99b5-ee5e59b91f78 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:23.410955 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adc464-1fd7-4061-99b5-ee5e59b91f78 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:23.416218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:23.416890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.417205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.417479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] PATH_INFO: `/v3/projects/550fb18df8234f6cb7d5fdf9770ce11e/users/c9efeb413ec245cfb83821f6b3433bbe/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.434612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=550fb18df8234f6cb7d5fdf9770ce11e, user_id=c9efeb413ec245cfb83821f6b3433bbe, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:23.437964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:23.451018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1208/2388] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:52:23 2022] PUT /identity/v3/projects/550fb18df8234f6cb7d5fdf9770ce11e/users/c9efeb413ec245cfb83821f6b3433bbe/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:52:23.456055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94965-48e3-43bb-824c-b860c1b640ec None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.456339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94965-48e3-43bb-824c-b860c1b640ec None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.456608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e94965-48e3-43bb-824c-b860c1b640ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.475259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d1e94965-48e3-43bb-824c-b860c1b640ec None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:23.476436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d1e94965-48e3-43bb-824c-b860c1b640ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:23.535036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1181/2389] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:23.549872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:23.551611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:23.607543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f541-7f8f-44b8-8a91-6c701f5352ee admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:23.611199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6d4dc-31ad-4cb1-9d83-d4145ab99fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:23.611803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6d4dc-31ad-4cb1-9d83-d4145ab99fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:23.612083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6d4dc-31ad-4cb1-9d83-d4145ab99fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:23.612354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6d4dc-31ad-4cb1-9d83-d4145ab99fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:23.621690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6d4dc-31ad-4cb1-9d83-d4145ab99fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:23.623639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6d4dc-31ad-4cb1-9d83-d4145ab99fc0 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:23.632899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1209/2390] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:52:28.168160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b70de-8271-4833-8390-aae55ad5cedc None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:28.168663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b70de-8271-4833-8390-aae55ad5cedc None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:28.168761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b70de-8271-4833-8390-aae55ad5cedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:28.210623 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-c42b70de-8271-4833-8390-aae55ad5cedc None None] MFA Rules not processed for user `c9efeb413ec245cfb83821f6b3433bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:28.211822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c42b70de-8271-4833-8390-aae55ad5cedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:28.297264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1182/2391] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:28.301131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6d4dc-31ad-4cb1-9d83-d4145ab99fc0 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:28.311721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6d4dc-31ad-4cb1-9d83-d4145ab99fc0 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:28.318617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfeae8-c1fa-47cf-80b2-0052cff6e1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:28.319260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfeae8-c1fa-47cf-80b2-0052cff6e1ee admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:28.319586 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfeae8-c1fa-47cf-80b2-0052cff6e1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:28.319978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfeae8-c1fa-47cf-80b2-0052cff6e1ee admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:28.320935 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfeae8-c1fa-47cf-80b2-0052cff6e1ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:28.323519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfeae8-c1fa-47cf-80b2-0052cff6e1ee admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:28.362277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1210/2392] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:52:28 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:52:28.366111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b70de-8271-4833-8390-aae55ad5cedc None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:28.404568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b70de-8271-4833-8390-aae55ad5cedc None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:28.443446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:28.480626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe2add-dbc4-452e-b38b-aa3615a0c8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:28.481269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe2add-dbc4-452e-b38b-aa3615a0c8d1 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:28.481703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe2add-dbc4-452e-b38b-aa3615a0c8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:28.481874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe2add-dbc4-452e-b38b-aa3615a0c8d1 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:28.483201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe2add-dbc4-452e-b38b-aa3615a0c8d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:28.485316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe2add-dbc4-452e-b38b-aa3615a0c8d1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:28.509926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1183/2393] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:52:28 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:52:28.514600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfeae8-c1fa-47cf-80b2-0052cff6e1ee admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:28.525386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfeae8-c1fa-47cf-80b2-0052cff6e1ee admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:28.530428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452c805-cda8-46e3-b3c7-71367b504918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:28.531037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452c805-cda8-46e3-b3c7-71367b504918 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:28.531428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452c805-cda8-46e3-b3c7-71367b504918 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:28.531570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452c805-cda8-46e3-b3c7-71367b504918 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:28.532446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452c805-cda8-46e3-b3c7-71367b504918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:28.534374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452c805-cda8-46e3-b3c7-71367b504918 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:28.552972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1211/2394] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:52:28.554468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe2add-dbc4-452e-b38b-aa3615a0c8d1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:28.566028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe2add-dbc4-452e-b38b-aa3615a0c8d1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:28.571296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c7ef3-6315-4daa-948d-a0f7534c5d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:28.571973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c7ef3-6315-4daa-948d-a0f7534c5d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:28.572255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c7ef3-6315-4daa-948d-a0f7534c5d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:28.572530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c7ef3-6315-4daa-948d-a0f7534c5d71 admin admin] PATH_INFO: `/v3/projects/d4665cf1f3a74bde87f7eabf701f4d22/users/c38f6f8c720045ce8200473b368fa79f/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:28.590262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c7ef3-6315-4daa-948d-a0f7534c5d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4665cf1f3a74bde87f7eabf701f4d22, user_id=c38f6f8c720045ce8200473b368fa79f, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:28.592713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c7ef3-6315-4daa-948d-a0f7534c5d71 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:28.608104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1184/2395] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:52:28 2022] PUT /identity/v3/projects/d4665cf1f3a74bde87f7eabf701f4d22/users/c38f6f8c720045ce8200473b368fa79f/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:28.612136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452c805-cda8-46e3-b3c7-71367b504918 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:28.622564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452c805-cda8-46e3-b3c7-71367b504918 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:28.687667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:28.688384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:28.688810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:28.689186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:28.689932 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:28.691604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:28.705528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1212/2396] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:52:28.709598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c7ef3-6315-4daa-948d-a0f7534c5d71 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:28.720940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c7ef3-6315-4daa-948d-a0f7534c5d71 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:28.734667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498a61b-75f6-4b25-b228-1f4630d86714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:28.735351 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a61b-75f6-4b25-b228-1f4630d86714 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:28.735634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a61b-75f6-4b25-b228-1f4630d86714 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:28.735904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498a61b-75f6-4b25-b228-1f4630d86714 admin admin] PATH_INFO: `/v3/projects/d4665cf1f3a74bde87f7eabf701f4d22/users/c38f6f8c720045ce8200473b368fa79f/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:28.739431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498a61b-75f6-4b25-b228-1f4630d86714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4665cf1f3a74bde87f7eabf701f4d22, user_id=c38f6f8c720045ce8200473b368fa79f, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:28.742364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498a61b-75f6-4b25-b228-1f4630d86714 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:28.754837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1185/2397] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:52:28 2022] PUT /identity/v3/projects/d4665cf1f3a74bde87f7eabf701f4d22/users/c38f6f8c720045ce8200473b368fa79f/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:32.475607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:32.478081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:32.493706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ef80d-2f4e-49df-ad3d-13f427363999 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:32.549588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:32.588443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:32.589717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:32.590104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:32.590485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] PATH_INFO: `/v3/users/656b76745b374947ba6cac54c70fd9bc` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:32.593347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=656b76745b374947ba6cac54c70fd9bc)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:32.595431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:32.667746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '656b76745b374947ba6cac54c70fd9bc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:32.671355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '656b76745b374947ba6cac54c70fd9bc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:32.690207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '656b76745b374947ba6cac54c70fd9bc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:32.701450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '656b76745b374947ba6cac54c70fd9bc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:32.702080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '656b76745b374947ba6cac54c70fd9bc'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:32.717172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1213/2398] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:52:32 2022] DELETE /identity/v3/users/656b76745b374947ba6cac54c70fd9bc => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:32.721406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80886fa-d3c3-4339-a1c4-486c3a7476ea None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:32.721874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80886fa-d3c3-4339-a1c4-486c3a7476ea None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:32.721957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80886fa-d3c3-4339-a1c4-486c3a7476ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:32.744010 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f80886fa-d3c3-4339-a1c4-486c3a7476ea None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:32.745861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f80886fa-d3c3-4339-a1c4-486c3a7476ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:32.867599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1186/2399] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:32.885032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:32.886721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:32.985262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed561fdd-37ed-4b8f-910f-d43617f14526 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:32.991349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ef9d1-4de2-4a27-a069-9f93e44dbbfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:32.992557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ef9d1-4de2-4a27-a069-9f93e44dbbfe service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:32.993296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ef9d1-4de2-4a27-a069-9f93e44dbbfe service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:32.993848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ef9d1-4de2-4a27-a069-9f93e44dbbfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:33.020881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ef9d1-4de2-4a27-a069-9f93e44dbbfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:33.024244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ef9d1-4de2-4a27-a069-9f93e44dbbfe service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:33.037348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1214/2400] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:52:33.666475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80886fa-d3c3-4339-a1c4-486c3a7476ea None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:33.667925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f80886fa-d3c3-4339-a1c4-486c3a7476ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:33.690793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80886fa-d3c3-4339-a1c4-486c3a7476ea None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:33.696210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:33.696854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:33.697465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:33.697855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] PATH_INFO: `/v3/projects/affdf63b83a4457da416c9442418e6e4` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:33.699194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] RBAC: Authorizing `identity:delete_project(project_id=affdf63b83a4457da416c9442418e6e4)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:33.702664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:33.762663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'affdf63b83a4457da416c9442418e6e4', 'request_id': 'req-cda5efc7-cfbb-4525-bb51-d42fad4808db'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:33.771479 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'affdf63b83a4457da416c9442418e6e4', 'request_id': 'req-cda5efc7-cfbb-4525-bb51-d42fad4808db'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:33.780243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'affdf63b83a4457da416c9442418e6e4', 'request_id': 'req-cda5efc7-cfbb-4525-bb51-d42fad4808db'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:33.804725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] The token cache is being invalidate because project affdf63b83a4457da416c9442418e6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:52:33.805400 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:33.806251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-cda5efc7-cfbb-4525-bb51-d42fad4808db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:52:33.808515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1187/2401] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:52:33 2022] DELETE /identity/v3/projects/affdf63b83a4457da416c9442418e6e4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:33.815589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02128f9c-70ac-4c60-ba0b-fe021e430720 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:33.816105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02128f9c-70ac-4c60-ba0b-fe021e430720 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:33.816105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02128f9c-70ac-4c60-ba0b-fe021e430720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:33.842830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-02128f9c-70ac-4c60-ba0b-fe021e430720 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:33.844821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-02128f9c-70ac-4c60-ba0b-fe021e430720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:33.920913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:33.936389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1215/2402] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:33.943137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0708-fedb-4a77-923f-f4b47a3feef4 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:33.943427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0708-fedb-4a77-923f-f4b47a3feef4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:33.943698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c0708-fedb-4a77-923f-f4b47a3feef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:33.954560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:33.968116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-642c0708-fedb-4a77-923f-f4b47a3feef4 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:33.969373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-642c0708-fedb-4a77-923f-f4b47a3feef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:33.985335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02128f9c-70ac-4c60-ba0b-fe021e430720 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:33.985525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1188/2403] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:33.997247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02128f9c-70ac-4c60-ba0b-fe021e430720 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:34.002735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:34.003379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.003653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.003921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.004563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:52:34.004876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:34.006493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:34.025362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1216/2404] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:52:34.029993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c0708-fedb-4a77-923f-f4b47a3feef4 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:34.043876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c0708-fedb-4a77-923f-f4b47a3feef4 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:34.052660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77601c-0aca-40f0-8f56-8f0b794550b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:34.053729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77601c-0aca-40f0-8f56-8f0b794550b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.057261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77601c-0aca-40f0-8f56-8f0b794550b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.057691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77601c-0aca-40f0-8f56-8f0b794550b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.058446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77601c-0aca-40f0-8f56-8f0b794550b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:34.060599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77601c-0aca-40f0-8f56-8f0b794550b3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:34.109276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1189/2405] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:52:34 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:52:34.111535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:34.127234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c48c5-3fde-45e2-b1fa-3a7c993601cf admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:34.215194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160140-7295-4c9a-92d2-aff0a5a7baff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:34.215913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160140-7295-4c9a-92d2-aff0a5a7baff admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.216604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160140-7295-4c9a-92d2-aff0a5a7baff admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.217213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160140-7295-4c9a-92d2-aff0a5a7baff admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.218352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46160140-7295-4c9a-92d2-aff0a5a7baff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:34.241381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46160140-7295-4c9a-92d2-aff0a5a7baff admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:34.276753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1217/2406] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:52:34 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:52:34.279871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0331-fd6a-40d3-b8d0-43ed893f3529 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.280158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0331-fd6a-40d3-b8d0-43ed893f3529 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.280513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e0331-fd6a-40d3-b8d0-43ed893f3529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.286140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160140-7295-4c9a-92d2-aff0a5a7baff admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:34.299717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160140-7295-4c9a-92d2-aff0a5a7baff admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:34.313098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96338376-b433-4f13-92b4-3a1e2c176359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:34.313726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338376-b433-4f13-92b4-3a1e2c176359 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.314006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338376-b433-4f13-92b4-3a1e2c176359 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.314277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96338376-b433-4f13-92b4-3a1e2c176359 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.314928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96338376-b433-4f13-92b4-3a1e2c176359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:34.317225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96338376-b433-4f13-92b4-3a1e2c176359 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:34.318019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-523e0331-fd6a-40d3-b8d0-43ed893f3529 None None] MFA Rules not processed for user `c38f6f8c720045ce8200473b368fa79f`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:34.319590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-523e0331-fd6a-40d3-b8d0-43ed893f3529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:34.333875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1218/2407] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:52:34.338254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96338376-b433-4f13-92b4-3a1e2c176359 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:34.358057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96338376-b433-4f13-92b4-3a1e2c176359 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:34.369832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c23063-61e7-49f0-b033-06000f5358a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:34.370974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c23063-61e7-49f0-b033-06000f5358a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.371454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c23063-61e7-49f0-b033-06000f5358a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.371928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c23063-61e7-49f0-b033-06000f5358a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.372636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c23063-61e7-49f0-b033-06000f5358a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:34.376553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c23063-61e7-49f0-b033-06000f5358a4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:34.401242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1219/2408] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:52:34.403063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c23063-61e7-49f0-b033-06000f5358a4 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:34.416143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c23063-61e7-49f0-b033-06000f5358a4 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:34.427040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e770b-16bf-4928-af56-ac4dd4f572af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:34.427968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e770b-16bf-4928-af56-ac4dd4f572af admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.428245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e770b-16bf-4928-af56-ac4dd4f572af admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.428526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e770b-16bf-4928-af56-ac4dd4f572af admin admin] PATH_INFO: `/v3/projects/bf6eb96014394a1bbd7838dc2eb03c13/users/efef230f10394d5eb31976cd0d693eda/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.431348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1190/2409] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:34.446270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e0331-fd6a-40d3-b8d0-43ed893f3529 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:34.448532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-523e0331-fd6a-40d3-b8d0-43ed893f3529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:34.453813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e770b-16bf-4928-af56-ac4dd4f572af admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6eb96014394a1bbd7838dc2eb03c13, user_id=efef230f10394d5eb31976cd0d693eda, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:34.456801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e770b-16bf-4928-af56-ac4dd4f572af admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:34.472580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1220/2410] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:52:34 2022] PUT /identity/v3/projects/bf6eb96014394a1bbd7838dc2eb03c13/users/efef230f10394d5eb31976cd0d693eda/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:52:34.477525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14013cd-f3fb-406d-a738-c03f7287c9b3 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.477839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14013cd-f3fb-406d-a738-c03f7287c9b3 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.478130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14013cd-f3fb-406d-a738-c03f7287c9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.500475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a14013cd-f3fb-406d-a738-c03f7287c9b3 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:34.502102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a14013cd-f3fb-406d-a738-c03f7287c9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:34.545831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e0331-fd6a-40d3-b8d0-43ed893f3529 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:34.594122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1221/2411] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:52:34.602610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14013cd-f3fb-406d-a738-c03f7287c9b3 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:34.604054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a14013cd-f3fb-406d-a738-c03f7287c9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:34.617994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:34.634075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:34.634700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.634985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.635755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.638030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:34.642163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:34.678239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14013cd-f3fb-406d-a738-c03f7287c9b3 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:34.681981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967d8be-a967-4874-a597-2153e71741f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:34.682602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967d8be-a967-4874-a597-2153e71741f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:34.682882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967d8be-a967-4874-a597-2153e71741f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:34.683156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967d8be-a967-4874-a597-2153e71741f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:34.693187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967d8be-a967-4874-a597-2153e71741f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:34.694961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967d8be-a967-4874-a597-2153e71741f7 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:34.712059 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:34.713858 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:34.714320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1222/2412] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:52:34.725106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1191/2413] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:52:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:52:35.937328 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5320b4-f7e9-4cab-b296-110075c98871 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:35.938457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5320b4-f7e9-4cab-b296-110075c98871 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:35.938847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5320b4-f7e9-4cab-b296-110075c98871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:35.992226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-3d5320b4-f7e9-4cab-b296-110075c98871 None None] MFA Rules not processed for user `efef230f10394d5eb31976cd0d693eda`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:35.993749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3d5320b4-f7e9-4cab-b296-110075c98871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:36.108482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1223/2414] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:36.119568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:36.121755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:36.168087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae9711-2926-40af-bcef-3dd29a7748c4 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:36.168479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae9711-2926-40af-bcef-3dd29a7748c4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:36.170063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae9711-2926-40af-bcef-3dd29a7748c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:36.189282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-03ae9711-2926-40af-bcef-3dd29a7748c4 None None] MFA Rules not processed for user `c9efeb413ec245cfb83821f6b3433bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:36.190621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-03ae9711-2926-40af-bcef-3dd29a7748c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:36.202067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c750f27-e00d-422d-9443-d986c176a940 service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:36.202783 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1224/2415] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:52:36.206165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711331c1-7d06-48c5-b046-aaf51c8ebf5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:36.206819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711331c1-7d06-48c5-b046-aaf51c8ebf5f service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:36.207102 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711331c1-7d06-48c5-b046-aaf51c8ebf5f service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:36.207403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711331c1-7d06-48c5-b046-aaf51c8ebf5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:36.219192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711331c1-7d06-48c5-b046-aaf51c8ebf5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:36.222071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711331c1-7d06-48c5-b046-aaf51c8ebf5f service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:36.232281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1192/2416] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:52:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:52:36.653574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae9711-2926-40af-bcef-3dd29a7748c4 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:52:36.667081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae9711-2926-40af-bcef-3dd29a7748c4 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:52:36.673451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:52:36.675124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:36.675402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:36.675723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:36.677916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:36.768782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:52:36.770875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:52:36.786843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1225/2417] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:52:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:52:37.799287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c9738-968b-4986-92b1-e8e62ee0e39b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:52:37.799834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c9738-968b-4986-92b1-e8e62ee0e39b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:52:37.800029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c9738-968b-4986-92b1-e8e62ee0e39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:52:37.850120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a99c9738-968b-4986-92b1-e8e62ee0e39b None None] MFA Rules not processed for user `efef230f10394d5eb31976cd0d693eda`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:52:37.851370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a99c9738-968b-4986-92b1-e8e62ee0e39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:52:37.873755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1193/2418] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:52:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:52:46.779876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=78 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:52:47.829325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=59 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:53:21.338322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:53:21.347093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1855c9-b4ce-445e-a03a-5ff3fb8887cc service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:53:21.350958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:53:21.351559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:53:21.351838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:53:21.352109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:53:21.411119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:53:21.412526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:53:21.422285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1226/2419] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:53:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:53:27.445800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3eb1c-6d17-4e6b-9d50-58c35e99a653 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:53:27.446303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3eb1c-6d17-4e6b-9d50-58c35e99a653 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:53:27.446404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3eb1c-6d17-4e6b-9d50-58c35e99a653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:53:27.465837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-69a3eb1c-6d17-4e6b-9d50-58c35e99a653 None None] MFA Rules not processed for user `04b569b91d1045dc8257c8aaa868842a`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:53:27.467006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-69a3eb1c-6d17-4e6b-9d50-58c35e99a653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:53:27.522283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1194/2420] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:53:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:53:27.559314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:53:27.560690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:53:27.616269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc6679d-ee5f-449a-a70b-944b7769d77c service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:53:27.620093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029447e782484e8f8a704b683ede1819', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:53:27.621208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:53:27.621481 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:53:27.621740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:53:27.623374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:53:27.636959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:53:27.638375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:53:27.649009 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1227/2421] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:53:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:53:28.250201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3eb1c-6d17-4e6b-9d50-58c35e99a653 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:53:28.265320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3eb1c-6d17-4e6b-9d50-58c35e99a653 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:53:28.267575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1f147-a0e8-44e3-be58-6f58fd151c29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:53:28.268270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1f147-a0e8-44e3-be58-6f58fd151c29 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:53:28.268608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1f147-a0e8-44e3-be58-6f58fd151c29 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:53:28.268942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1f147-a0e8-44e3-be58-6f58fd151c29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:53:28.270804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d6e1f147-a0e8-44e3-be58-6f58fd151c29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:53:28.284641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1f147-a0e8-44e3-be58-6f58fd151c29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:53:28.286087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1f147-a0e8-44e3-be58-6f58fd151c29 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:53:28.297416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1195/2422] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:53:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:53:37.644267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=56 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:53:38.292685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=31 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:53:44.861990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:53:44.874426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e97b9-2c2c-4b52-b187-3a31633cc025 service glance] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:53:44.880988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:53:44.881650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:53:44.881869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:53:44.882135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:53:44.883852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:53:44.951666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:53:44.953334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:53:44.962785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1228/2423] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:53:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:53:54.958360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=28 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:54:16.130241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7259306-d0f9-4453-8a24-af92b5883219 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:16.130748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7259306-d0f9-4453-8a24-af92b5883219 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:16.130846 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7259306-d0f9-4453-8a24-af92b5883219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:16.149985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-c7259306-d0f9-4453-8a24-af92b5883219 None None] MFA Rules not processed for user `efef230f10394d5eb31976cd0d693eda`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:54:16.151706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c7259306-d0f9-4453-8a24-af92b5883219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:16.160368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1196/2424] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:54:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:54:16.385304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:54:16.395177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdeb687-86e9-4418-9e9f-c787a568fd7a service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:54:16.398947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:54:16.399609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:16.400491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:16.400838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:16.402721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:16.412543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:54:16.413972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:54:16.422354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1229/2425] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:54:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:54:16.732206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7259306-d0f9-4453-8a24-af92b5883219 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:54:16.733760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c7259306-d0f9-4453-8a24-af92b5883219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:16.803871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7259306-d0f9-4453-8a24-af92b5883219 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:54:16.809497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:54:16.810097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:16.810374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:16.810645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:16.812364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:16.823446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:54:16.824993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:54:16.834735 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1197/2426] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:54:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:54:18.928137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:54:18.937248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2763dfd-3dc9-4680-a528-3bb7b47df0d0 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:54:18.940931 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:54:18.941583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:18.941857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:18.942122 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:18.943834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:18.954631 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:54:18.956043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:54:18.966356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1230/2427] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:54:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:54:19.715011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:54:19.717165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:19.734917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723ec25-cded-4fab-a4f6-400239f83875 service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:54:19.740512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02697582-1c21-4342-a12b-ef9f5ef9356a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:54:19.741140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697582-1c21-4342-a12b-ef9f5ef9356a service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:19.741425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697582-1c21-4342-a12b-ef9f5ef9356a service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:19.741697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697582-1c21-4342-a12b-ef9f5ef9356a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:19.743408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-02697582-1c21-4342-a12b-ef9f5ef9356a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:19.756587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02697582-1c21-4342-a12b-ef9f5ef9356a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:54:19.758203 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02697582-1c21-4342-a12b-ef9f5ef9356a service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:54:19.769349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1198/2428] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:54:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:54:27.987954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:54:27.991004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92ae1-b24d-42f9-98f7-5cba23cf16f7 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:27.991443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92ae1-b24d-42f9-98f7-5cba23cf16f7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:27.991560 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f92ae1-b24d-42f9-98f7-5cba23cf16f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:28.001496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0164386-00b5-4d6a-b8b9-e965696c386b service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:54:28.006987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:54:28.007674 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:28.007947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:28.008271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:28.010555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:28.010669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-c1f92ae1-b24d-42f9-98f7-5cba23cf16f7 None None] MFA Rules not processed for user `3153c9ff011f4ab8958fc49795e6f8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:54:28.011767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c1f92ae1-b24d-42f9-98f7-5cba23cf16f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:28.018965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1199/2429] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:54:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:54:28.079407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:54:28.080975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:54:28.090751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1231/2430] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:54:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:54:30.944665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f92ae1-b24d-42f9-98f7-5cba23cf16f7 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:54:30.958700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f92ae1-b24d-42f9-98f7-5cba23cf16f7 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:54:30.964455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:54:30.965054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:30.965361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:30.965630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:30.967327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:30.980335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:54:30.981976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:54:30.992460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1200/2431] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:54:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:54:34.165628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:54:34.176785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce024b-d0e5-4d81-b4a5-251b0174192f service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:54:34.184016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:54:34.184838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:54:34.185309 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:54:34.185758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:54:34.187708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:54:34.217631 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:54:34.219245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:54:34.230986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1232/2432] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:54:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:54:40.988077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=50 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:54:44.225542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=46 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:55:33.785482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:33.797716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c63c4-d4e2-4a65-afef-7001bd64f1cc service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:33.809410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:33.810029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:33.810301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:33.810563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:33.812522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:33.825220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:33.827034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:33.841411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1201/2433] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:55:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:55:33.842780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:33.856107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3f285-7430-4fbf-b07b-b77082701c64 service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:33.861401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673599bc-2283-487c-b040-92627b167947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:33.862028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673599bc-2283-487c-b040-92627b167947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:33.862339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673599bc-2283-487c-b040-92627b167947 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:33.862636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673599bc-2283-487c-b040-92627b167947 admin admin] PATH_INFO: `/v3/users/efef230f10394d5eb31976cd0d693eda` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:33.863753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673599bc-2283-487c-b040-92627b167947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efef230f10394d5eb31976cd0d693eda)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:33.866198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673599bc-2283-487c-b040-92627b167947 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:33.913476 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-673599bc-2283-487c-b040-92627b167947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efef230f10394d5eb31976cd0d693eda'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:33.914128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-673599bc-2283-487c-b040-92627b167947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efef230f10394d5eb31976cd0d693eda'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:33.930234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-673599bc-2283-487c-b040-92627b167947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efef230f10394d5eb31976cd0d693eda'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:33.937494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-673599bc-2283-487c-b040-92627b167947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efef230f10394d5eb31976cd0d693eda'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:33.938119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-673599bc-2283-487c-b040-92627b167947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efef230f10394d5eb31976cd0d693eda'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:33.951604 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1233/2434] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:55:33 2022] DELETE /identity/v3/users/efef230f10394d5eb31976cd0d693eda => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:55:34.318215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:34.331498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c460f97-544f-4e46-841d-1a45aff7289b service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:34.390613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:34.391239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.391520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.391793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] PATH_INFO: `/v3/projects/bf6eb96014394a1bbd7838dc2eb03c13` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.392909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf6eb96014394a1bbd7838dc2eb03c13)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:34.394839 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:34.443267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6eb96014394a1bbd7838dc2eb03c13', 'request_id': 'req-8ebe7973-80e2-47b5-a464-149feb84cea7'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:34.448683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6eb96014394a1bbd7838dc2eb03c13', 'request_id': 'req-8ebe7973-80e2-47b5-a464-149feb84cea7'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:34.454728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6eb96014394a1bbd7838dc2eb03c13', 'request_id': 'req-8ebe7973-80e2-47b5-a464-149feb84cea7'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:34.467198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] The token cache is being invalidate because project bf6eb96014394a1bbd7838dc2eb03c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:55:34.467484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:34.468105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-8ebe7973-80e2-47b5-a464-149feb84cea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:34.469811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1202/2435] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:55:34 2022] DELETE /identity/v3/projects/bf6eb96014394a1bbd7838dc2eb03c13 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:55:34.475200 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83ea0-b661-46c3-8e48-8496f66909eb None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.475485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83ea0-b661-46c3-8e48-8496f66909eb None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.475756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c83ea0-b661-46c3-8e48-8496f66909eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.495152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f6c83ea0-b661-46c3-8e48-8496f66909eb None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:55:34.496343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f6c83ea0-b661-46c3-8e48-8496f66909eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:34.553484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1234/2436] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:55:34.561353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4317469-1b2c-4b11-ae9b-1d89b5593387 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.561591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4317469-1b2c-4b11-ae9b-1d89b5593387 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.561874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4317469-1b2c-4b11-ae9b-1d89b5593387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.581374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-f4317469-1b2c-4b11-ae9b-1d89b5593387 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:55:34.582589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f4317469-1b2c-4b11-ae9b-1d89b5593387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:34.590907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1203/2437] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:55:34.594931 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c83ea0-b661-46c3-8e48-8496f66909eb None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:34.604957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c83ea0-b661-46c3-8e48-8496f66909eb None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:34.609510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:34.610100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.610382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.610651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.611289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:55:34.611590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:34.613187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:34.624842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1235/2438] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:55:34.628808 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4317469-1b2c-4b11-ae9b-1d89b5593387 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:34.637903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4317469-1b2c-4b11-ae9b-1d89b5593387 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:34.642495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0dc4-add5-4567-a640-adf141247deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:34.643076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0dc4-add5-4567-a640-adf141247deb admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.643355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0dc4-add5-4567-a640-adf141247deb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.643629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0dc4-add5-4567-a640-adf141247deb admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.644259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d0dc4-add5-4567-a640-adf141247deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:34.646737 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d0dc4-add5-4567-a640-adf141247deb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:34.673167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1204/2439] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:55:34 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:55:34.677104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:34.687449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb63921-5525-4ef7-a5c9-5d28941bbc2b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:34.741964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ceb60-a0fb-413f-b888-ad3ff54d423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:34.742557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ceb60-a0fb-413f-b888-ad3ff54d423f admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.742834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ceb60-a0fb-413f-b888-ad3ff54d423f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.743359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ceb60-a0fb-413f-b888-ad3ff54d423f admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.743983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ceb60-a0fb-413f-b888-ad3ff54d423f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:34.745922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ceb60-a0fb-413f-b888-ad3ff54d423f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:34.753960 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:55:34.771951 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1236/2440] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:55:34 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:55:34.773368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0dc4-add5-4567-a640-adf141247deb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:34.783424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0dc4-add5-4567-a640-adf141247deb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:34.787954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187bbe-d281-4454-b88a-7edc2b853c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:34.788536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187bbe-d281-4454-b88a-7edc2b853c05 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.788792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187bbe-d281-4454-b88a-7edc2b853c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.789079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187bbe-d281-4454-b88a-7edc2b853c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.789701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed187bbe-d281-4454-b88a-7edc2b853c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:34.791270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed187bbe-d281-4454-b88a-7edc2b853c05 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:34.800863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1205/2441] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:55:34.804574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ceb60-a0fb-413f-b888-ad3ff54d423f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:34.814407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ceb60-a0fb-413f-b888-ad3ff54d423f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:34.819469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d81a14-0eb5-4f73-8cb5-38c36c0f6618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:34.820054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d81a14-0eb5-4f73-8cb5-38c36c0f6618 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.820332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d81a14-0eb5-4f73-8cb5-38c36c0f6618 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.820605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d81a14-0eb5-4f73-8cb5-38c36c0f6618 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.821277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d81a14-0eb5-4f73-8cb5-38c36c0f6618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:34.822846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d81a14-0eb5-4f73-8cb5-38c36c0f6618 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:34.833253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1237/2442] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:55:34.837994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187bbe-d281-4454-b88a-7edc2b853c05 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:34.847627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187bbe-d281-4454-b88a-7edc2b853c05 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:34.852176 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:34.852812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.853126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.853403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] PATH_INFO: `/v3/projects/185a5af6a847453d85717c81a20c4375/users/e0b89fb63b5340ee961cc7a60d710c53/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.869128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=185a5af6a847453d85717c81a20c4375, user_id=e0b89fb63b5340ee961cc7a60d710c53, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:34.871739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:34.884873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1206/2443] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:55:34 2022] PUT /identity/v3/projects/185a5af6a847453d85717c81a20c4375/users/e0b89fb63b5340ee961cc7a60d710c53/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:55:34.895975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b419deb-efd0-4b04-a739-e98d96cd4f8d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:34.897109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b419deb-efd0-4b04-a739-e98d96cd4f8d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:34.897218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b419deb-efd0-4b04-a739-e98d96cd4f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:34.926112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-0b419deb-efd0-4b04-a739-e98d96cd4f8d None None] MFA Rules not processed for user `e0b89fb63b5340ee961cc7a60d710c53`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:55:34.927361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0b419deb-efd0-4b04-a739-e98d96cd4f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:34.990928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1238/2444] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:55:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:55:35.001876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:35.003534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:35.017714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:55:35.061840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c50238-b58d-4bbd-8372-ab94f7f5f77d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:35.065559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3d901-1793-4302-a862-1268f79957c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:35.066119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3d901-1793-4302-a862-1268f79957c0 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:35.066395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3d901-1793-4302-a862-1268f79957c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:35.066665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3d901-1793-4302-a862-1268f79957c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:35.074888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3d901-1793-4302-a862-1268f79957c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:35.076295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3d901-1793-4302-a862-1268f79957c0 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:35.085092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1207/2445] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:55:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:55:44.987403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:55:45.080969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=20 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:55:55.142496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162492d-b2e4-4c30-8bc4-032de8ffad2f None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:55.142997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162492d-b2e4-4c30-8bc4-032de8ffad2f None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:55.143105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162492d-b2e4-4c30-8bc4-032de8ffad2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:55.162834 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-8162492d-b2e4-4c30-8bc4-032de8ffad2f None None] MFA Rules not processed for user `fa5465b818f34778a1d51278653488ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:55:55.164045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8162492d-b2e4-4c30-8bc4-032de8ffad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:55.219028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1239/2446] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:55:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:55:57.507034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3d901-1793-4302-a862-1268f79957c0 service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:57.576431 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3d901-1793-4302-a862-1268f79957c0 service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:57.580997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:57.581653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:57.581930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:57.582198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] PATH_INFO: `/v3/users/e0b89fb63b5340ee961cc7a60d710c53` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:57.583347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b89fb63b5340ee961cc7a60d710c53)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:57.585275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:57.635585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b89fb63b5340ee961cc7a60d710c53'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:57.636236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b89fb63b5340ee961cc7a60d710c53'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:57.644108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b89fb63b5340ee961cc7a60d710c53'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:57.657197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b89fb63b5340ee961cc7a60d710c53'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:57.664828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b89fb63b5340ee961cc7a60d710c53'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:57.670876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1208/2447] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:55:57 2022] DELETE /identity/v3/users/e0b89fb63b5340ee961cc7a60d710c53 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:55:57.675394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f66c2a-f76e-4bf5-9db9-e20d1584179d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:57.675859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f66c2a-f76e-4bf5-9db9-e20d1584179d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:57.676012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f66c2a-f76e-4bf5-9db9-e20d1584179d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:57.695080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a3f66c2a-f76e-4bf5-9db9-e20d1584179d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:55:57.696487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a3f66c2a-f76e-4bf5-9db9-e20d1584179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:57.755763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1240/2448] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:55:57.767481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:57.769125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:57.829424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e011fa-255d-498a-b358-a7e3497bed86 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:57.833628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:57.834234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:57.834510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:57.834780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:57.843860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:57.845313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:57.854806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1209/2449] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:55:58.331639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f66c2a-f76e-4bf5-9db9-e20d1584179d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:55:58.333386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a3f66c2a-f76e-4bf5-9db9-e20d1584179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:55:58.349866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f66c2a-f76e-4bf5-9db9-e20d1584179d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:55:58.355441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:55:58.356312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:55:58.356653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:55:58.357257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] PATH_INFO: `/v3/projects/185a5af6a847453d85717c81a20c4375` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:55:58.358520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=185a5af6a847453d85717c81a20c4375)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:55:58.360535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:55:58.457098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185a5af6a847453d85717c81a20c4375', 'request_id': 'req-d87361f8-c643-420a-9095-4317a258cdf4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:58.464687 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185a5af6a847453d85717c81a20c4375', 'request_id': 'req-d87361f8-c643-420a-9095-4317a258cdf4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:58.471281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185a5af6a847453d85717c81a20c4375', 'request_id': 'req-d87361f8-c643-420a-9095-4317a258cdf4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:58.487060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] The token cache is being invalidate because project 185a5af6a847453d85717c81a20c4375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:55:58.487353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:58.487997 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d87361f8-c643-420a-9095-4317a258cdf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:55:58.490386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1241/2450] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:55:58 2022] DELETE /identity/v3/projects/185a5af6a847453d85717c81a20c4375 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:56:07.850522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:08.295456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:08.296874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:08.370901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6db448-bdec-49c2-b797-854a07481b75 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:08.439449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:08.440060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:08.440338 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:08.440608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:08.443605 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:08.483973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:08.504925 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:08.506653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:08.516161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1210/2451] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:56:18.511942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=84 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:26.921648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bf869-a7e8-4253-aa40-3eb53aaac19b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:26.922634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bf869-a7e8-4253-aa40-3eb53aaac19b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:26.923154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bf869-a7e8-4253-aa40-3eb53aaac19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:26.965464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:26.967075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:26.967944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f93bf869-a7e8-4253-aa40-3eb53aaac19b None None] MFA Rules not processed for user `04b569b91d1045dc8257c8aaa868842a`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:26.969507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f93bf869-a7e8-4253-aa40-3eb53aaac19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:27.010394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1242/2452] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:56:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:56:27.131799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c37972-757b-4ff1-8d0b-8b26ff922dc8 service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:27.137992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:27.146864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:27.147218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:27.147580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:27.149774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:27.270230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:27.271662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:27.297288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1211/2453] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:56:27.483342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bf869-a7e8-4253-aa40-3eb53aaac19b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:27.485035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f93bf869-a7e8-4253-aa40-3eb53aaac19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:27.508837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bf869-a7e8-4253-aa40-3eb53aaac19b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:27.514967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:27.516205 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:27.517018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:27.517323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:27.520663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:27.537335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:27.539169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:27.551407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1243/2454] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:56:31.180611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:31.180611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:31.248373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:31.255184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:31.257381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:31.304208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bee227-63d2-48bb-a067-bd8bc143ebe0 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:31.310458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:31.311115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:31.311399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:31.311675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] PATH_INFO: `/v3/users/3153c9ff011f4ab8958fc49795e6f8f1` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:31.312987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3153c9ff011f4ab8958fc49795e6f8f1)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:31.315108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:31.345167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4e35c-5284-4d9f-aeab-6d78dca158be service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:31.353180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:31.354440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:31.354969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:31.355307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:31.358281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:31.390135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:31.391916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:31.395588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3153c9ff011f4ab8958fc49795e6f8f1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.396273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3153c9ff011f4ab8958fc49795e6f8f1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.411697 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1212/2455] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:56:31.434566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3153c9ff011f4ab8958fc49795e6f8f1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.449895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3153c9ff011f4ab8958fc49795e6f8f1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.450828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a7e9435b-a6b2-493a-b240-098be99c5128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3153c9ff011f4ab8958fc49795e6f8f1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.465357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1244/2456] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:56:31 2022] DELETE /identity/v3/users/3153c9ff011f4ab8958fc49795e6f8f1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:56:31.660446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:31.677330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047cdb8-accc-4376-bfb9-12e310708c87 service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:31.741632 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:31.742545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:31.742818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:31.743086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] PATH_INFO: `/v3/projects/6b35000faf844d758c12de9938c235a8` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:31.744193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b35000faf844d758c12de9938c235a8)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:31.746253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:31.794317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b35000faf844d758c12de9938c235a8', 'request_id': 'req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.800501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b35000faf844d758c12de9938c235a8', 'request_id': 'req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.807446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b35000faf844d758c12de9938c235a8', 'request_id': 'req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.821406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] The token cache is being invalidate because project 6b35000faf844d758c12de9938c235a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:56:31.821679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.822299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b3e7abc5-8048-40bc-b85b-5afb16b48b5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:56:31.823800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1213/2457] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:56:31 2022] DELETE /identity/v3/projects/6b35000faf844d758c12de9938c235a8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:56:31.829237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ffce-fa84-42e1-b5fa-88792ab77fdf None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:31.829802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ffce-fa84-42e1-b5fa-88792ab77fdf None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:31.830032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14ffce-fa84-42e1-b5fa-88792ab77fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:31.890121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-eb14ffce-fa84-42e1-b5fa-88792ab77fdf None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:31.891329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-eb14ffce-fa84-42e1-b5fa-88792ab77fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:31.942496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:31.955460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1245/2458] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:56:31.963869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8ccfb-2b9e-45c9-b1de-17bb7ba37b0d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:31.964247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8ccfb-2b9e-45c9-b1de-17bb7ba37b0d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:31.964601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8ccfb-2b9e-45c9-b1de-17bb7ba37b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:31.985721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-48f8ccfb-2b9e-45c9-b1de-17bb7ba37b0d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:31.987030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-48f8ccfb-2b9e-45c9-b1de-17bb7ba37b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:31.995369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1214/2459] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:56:31.999619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14ffce-fa84-42e1-b5fa-88792ab77fdf None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:32.010552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14ffce-fa84-42e1-b5fa-88792ab77fdf None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:32.015766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:32.016382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:32.016648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:32.016916 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:32.017790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:56:32.018103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:32.019727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:32.032459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1246/2460] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:56:32.037893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8ccfb-2b9e-45c9-b1de-17bb7ba37b0d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:32.048439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8ccfb-2b9e-45c9-b1de-17bb7ba37b0d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:32.054045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af9acb-bad8-4a26-ba01-4b9d7b3ae224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:32.054741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af9acb-bad8-4a26-ba01-4b9d7b3ae224 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:32.054927 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af9acb-bad8-4a26-ba01-4b9d7b3ae224 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:32.055196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af9acb-bad8-4a26-ba01-4b9d7b3ae224 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:32.056192 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af9acb-bad8-4a26-ba01-4b9d7b3ae224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:32.058125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af9acb-bad8-4a26-ba01-4b9d7b3ae224 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:32.090792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1215/2461] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:56:32 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:56:32.095016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:32.105801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a068b0-5816-47c4-a191-e7ece6952d7e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:32.175320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b235c25-ddf6-4152-937c-b57db6b76dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:32.176655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b235c25-ddf6-4152-937c-b57db6b76dad admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:32.176998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b235c25-ddf6-4152-937c-b57db6b76dad admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:32.177367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b235c25-ddf6-4152-937c-b57db6b76dad admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:32.178016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b235c25-ddf6-4152-937c-b57db6b76dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:32.180076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b235c25-ddf6-4152-937c-b57db6b76dad admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:32.206465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1247/2462] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:56:32 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:56:32.208460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af9acb-bad8-4a26-ba01-4b9d7b3ae224 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:32.223377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af9acb-bad8-4a26-ba01-4b9d7b3ae224 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:32.229041 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be3301-f71a-47f2-a8e5-61ac66ea1d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:32.229683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3301-f71a-47f2-a8e5-61ac66ea1d6c admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:32.229960 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3301-f71a-47f2-a8e5-61ac66ea1d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:32.230234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be3301-f71a-47f2-a8e5-61ac66ea1d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:32.230874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be3301-f71a-47f2-a8e5-61ac66ea1d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:32.232474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be3301-f71a-47f2-a8e5-61ac66ea1d6c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:32.244319 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1216/2463] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:56:32.248403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b235c25-ddf6-4152-937c-b57db6b76dad admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:32.259993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b235c25-ddf6-4152-937c-b57db6b76dad admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:32.264750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9e8f8-b830-431c-9641-8803d7abfa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:32.265416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9e8f8-b830-431c-9641-8803d7abfa80 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:32.265695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9e8f8-b830-431c-9641-8803d7abfa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:32.265962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9e8f8-b830-431c-9641-8803d7abfa80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:32.266595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9e8f8-b830-431c-9641-8803d7abfa80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:32.268172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9e8f8-b830-431c-9641-8803d7abfa80 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:32.282504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1248/2464] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:56:32.283755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be3301-f71a-47f2-a8e5-61ac66ea1d6c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:32.293703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be3301-f71a-47f2-a8e5-61ac66ea1d6c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:32.298222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:32.298856 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:32.299130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:32.299395 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] PATH_INFO: `/v3/projects/d2f935cd526946779ecce236c9d47f3f/users/eaaf03c8c05d4f598328ef5c05974e5e/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:32.302130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:32.318554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f935cd526946779ecce236c9d47f3f, user_id=eaaf03c8c05d4f598328ef5c05974e5e, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:32.321304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:32.340853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1217/2465] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:56:32 2022] PUT /identity/v3/projects/d2f935cd526946779ecce236c9d47f3f/users/eaaf03c8c05d4f598328ef5c05974e5e/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:56:32.342685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32f1b3-6962-4445-bc15-145a791c61a7 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:32.343077 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32f1b3-6962-4445-bc15-145a791c61a7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:32.343519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32f1b3-6962-4445-bc15-145a791c61a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:32.367533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-9f32f1b3-6962-4445-bc15-145a791c61a7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:32.368845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9f32f1b3-6962-4445-bc15-145a791c61a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:32.430247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1249/2466] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:56:32.440099 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:32.442482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:32.505301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5be0f-2c3c-473c-8f95-dff3dad7fc47 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:32.509583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438aa41-24a4-4dfa-b190-62275780dbe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:32.510631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438aa41-24a4-4dfa-b190-62275780dbe7 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:32.511142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438aa41-24a4-4dfa-b190-62275780dbe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:32.511652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438aa41-24a4-4dfa-b190-62275780dbe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:32.523777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438aa41-24a4-4dfa-b190-62275780dbe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:32.525269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438aa41-24a4-4dfa-b190-62275780dbe7 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:32.534975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1218/2467] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:56:33.026058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be927-46f2-4704-8217-6d2e75cf14c3 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:33.026470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be927-46f2-4704-8217-6d2e75cf14c3 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:33.026650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be927-46f2-4704-8217-6d2e75cf14c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:33.046740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-595be927-46f2-4704-8217-6d2e75cf14c3 None None] MFA Rules not processed for user `83f9dc8c41604f0e86eaf57202285cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:33.047971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-595be927-46f2-4704-8217-6d2e75cf14c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:33.111290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1250/2468] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:56:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:56:37.304373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1321d8-4ac2-49a3-8613-a0dfd771b9b0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:37.304956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1321d8-4ac2-49a3-8613-a0dfd771b9b0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:37.304956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1321d8-4ac2-49a3-8613-a0dfd771b9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:37.348306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-2c1321d8-4ac2-49a3-8613-a0dfd771b9b0 None None] MFA Rules not processed for user `eaaf03c8c05d4f598328ef5c05974e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:37.350135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2c1321d8-4ac2-49a3-8613-a0dfd771b9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:37.437571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1219/2469] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:56:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:56:37.455570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be927-46f2-4704-8217-6d2e75cf14c3 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:37.457659 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-595be927-46f2-4704-8217-6d2e75cf14c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:37.469851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:37.640546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be927-46f2-4704-8217-6d2e75cf14c3 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:37.658240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a23de1-5c73-4e9a-94af-bc953432917b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:37.659107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a23de1-5c73-4e9a-94af-bc953432917b service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:37.659490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a23de1-5c73-4e9a-94af-bc953432917b service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:37.659944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a23de1-5c73-4e9a-94af-bc953432917b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:37.669334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a23de1-5c73-4e9a-94af-bc953432917b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:37.671297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a23de1-5c73-4e9a-94af-bc953432917b service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:37.684214 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1251/2470] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:56:38.521196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1321d8-4ac2-49a3-8613-a0dfd771b9b0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:38.522806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2c1321d8-4ac2-49a3-8613-a0dfd771b9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:38.693380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1321d8-4ac2-49a3-8613-a0dfd771b9b0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:38.700838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2199d19-bff9-42af-ab24-42ec6c775fab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:38.701759 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2199d19-bff9-42af-ab24-42ec6c775fab service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:38.702205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2199d19-bff9-42af-ab24-42ec6c775fab service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:38.702795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2199d19-bff9-42af-ab24-42ec6c775fab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:38.705390 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c2199d19-bff9-42af-ab24-42ec6c775fab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:38.720181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2199d19-bff9-42af-ab24-42ec6c775fab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:38.722595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2199d19-bff9-42af-ab24-42ec6c775fab service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:38.743888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1220/2471] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:56:39.525236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e6c88-044a-45a2-891c-924c24d56bda None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:39.525768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e6c88-044a-45a2-891c-924c24d56bda None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:39.525768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592e6c88-044a-45a2-891c-924c24d56bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:39.552959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-592e6c88-044a-45a2-891c-924c24d56bda None None] MFA Rules not processed for user `eaaf03c8c05d4f598328ef5c05974e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:39.554313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-592e6c88-044a-45a2-891c-924c24d56bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:39.562080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1252/2472] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:56:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:56:44.961540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0905a9-8ebe-401b-8cbc-242cb2f5dcca None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:44.962531 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0905a9-8ebe-401b-8cbc-242cb2f5dcca None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:44.962880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0905a9-8ebe-401b-8cbc-242cb2f5dcca None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:44.964605 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1221/2473] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:56:44.970796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f088a9b-4c95-442f-b340-37cbf2005977 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:44.971276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f088a9b-4c95-442f-b340-37cbf2005977 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:44.971393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f088a9b-4c95-442f-b340-37cbf2005977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:45.018121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-0f088a9b-4c95-442f-b340-37cbf2005977 None None] MFA Rules not processed for user `341277e7e3144e5ea93fa657916821e1`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:45.020160 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-0f088a9b-4c95-442f-b340-37cbf2005977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:45.042224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1253/2474] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:56:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:56:45.071371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0905a9-8ebe-401b-8cbc-242cb2f5dcca None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:45.080820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:45.090954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0905a9-8ebe-401b-8cbc-242cb2f5dcca None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:45.097669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf9457-eb65-4c0e-b9e8-5c2c76520dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:45.098681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9457-eb65-4c0e-b9e8-5c2c76520dfc service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:45.099174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9457-eb65-4c0e-b9e8-5c2c76520dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:45.100351 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9457-eb65-4c0e-b9e8-5c2c76520dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:45.112707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf9457-eb65-4c0e-b9e8-5c2c76520dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:45.115407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf9457-eb65-4c0e-b9e8-5c2c76520dfc service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:45.129575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1222/2475] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:56:45.268817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85354fd9-fade-4e21-9661-0e95ffd571b7 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:45.269029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85354fd9-fade-4e21-9661-0e95ffd571b7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:45.269329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85354fd9-fade-4e21-9661-0e95ffd571b7 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:45.271150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1254/2476] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:56:45.277258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614ee1-3746-46ff-84ad-3905cebe338b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:45.277435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614ee1-3746-46ff-84ad-3905cebe338b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:45.277905 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614ee1-3746-46ff-84ad-3905cebe338b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:45.299066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7d614ee1-3746-46ff-84ad-3905cebe338b None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:56:45.300849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7d614ee1-3746-46ff-84ad-3905cebe338b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:45.310545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1223/2477] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:56:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:56:45.339297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85354fd9-fade-4e21-9661-0e95ffd571b7 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:56:45.341397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-85354fd9-fade-4e21-9661-0e95ffd571b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:56:45.359085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85354fd9-fade-4e21-9661-0e95ffd571b7 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:56:45.364341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c709b1f-4a6f-4b4e-b515-7246ca8a9825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:56:45.365296 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c709b1f-4a6f-4b4e-b515-7246ca8a9825 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:56:45.365851 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c709b1f-4a6f-4b4e-b515-7246ca8a9825 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:56:45.366322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c709b1f-4a6f-4b4e-b515-7246ca8a9825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:56:45.394850 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c709b1f-4a6f-4b4e-b515-7246ca8a9825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:56:45.396320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c709b1f-4a6f-4b4e-b515-7246ca8a9825 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:56:45.408688 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1255/2478] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:56:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:56:55.293570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=7 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:56:55.404079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=68 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:58:09.258336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d614ee1-3746-46ff-84ad-3905cebe338b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:09.259714 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7d614ee1-3746-46ff-84ad-3905cebe338b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:09.334337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d614ee1-3746-46ff-84ad-3905cebe338b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:09.337955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:09.338552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:09.338829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:09.339098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:09.340803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:09.399915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:09.401383 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:09.411122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1224/2479] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:58:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:58:19.406644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=57 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:58:21.480981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1dc98-acad-4100-ad7c-1cf928dd57c7 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:21.481543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1dc98-acad-4100-ad7c-1cf928dd57c7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:21.481638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1dc98-acad-4100-ad7c-1cf928dd57c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:21.501929 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2fb1dc98-acad-4100-ad7c-1cf928dd57c7 None None] MFA Rules not processed for user `fa5465b818f34778a1d51278653488ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:58:21.503172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2fb1dc98-acad-4100-ad7c-1cf928dd57c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:21.513124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1256/2480] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:58:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:58:27.219641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:27.228868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19f0a6-ae4e-4f5a-9f7e-85a03b3108ba service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:27.232447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6254e6-f9c7-4759-80e7-1d2635315253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:27.233030 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6254e6-f9c7-4759-80e7-1d2635315253 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:27.233336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6254e6-f9c7-4759-80e7-1d2635315253 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:27.233606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6254e6-f9c7-4759-80e7-1d2635315253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:27.235339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8d6254e6-f9c7-4759-80e7-1d2635315253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:27.249424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6254e6-f9c7-4759-80e7-1d2635315253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:27.250840 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6254e6-f9c7-4759-80e7-1d2635315253 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:27.261418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1225/2481] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:58:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:58:28.436949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1dc98-acad-4100-ad7c-1cf928dd57c7 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:28.448343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1dc98-acad-4100-ad7c-1cf928dd57c7 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:28.453322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:28.453958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:28.454236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:28.454602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:28.456375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:28.472103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:28.473596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:28.485277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1257/2482] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:58:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:58:37.256644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=6 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:58:38.481782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=9 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:58:39.808834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0aa9e-36a2-4ae2-8498-4a8e006b035b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:39.809378 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0aa9e-36a2-4ae2-8498-4a8e006b035b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:39.809485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0aa9e-36a2-4ae2-8498-4a8e006b035b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:39.829225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-71e0aa9e-36a2-4ae2-8498-4a8e006b035b None None] MFA Rules not processed for user `04b569b91d1045dc8257c8aaa868842a`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:58:39.830398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-71e0aa9e-36a2-4ae2-8498-4a8e006b035b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:39.889527 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1226/2483] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:58:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:58:42.591887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:42.601126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3fd6c-ca4a-4af2-a803-635fbac9b32e service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:42.604841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:42.605474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:42.605748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:42.606022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:42.608274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:42.670552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:42.671965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:42.681348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1258/2484] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:58:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:58:44.545292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0aa9e-36a2-4ae2-8498-4a8e006b035b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:44.555484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0aa9e-36a2-4ae2-8498-4a8e006b035b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:44.559426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:44.560447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:44.560713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:44.560991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:44.562798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:44.639132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:44.640555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:44.654140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1227/2485] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:58:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:58:50.298752 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:50.307399 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c523ab2-a787-40c7-ac96-81232071f8e5 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:50.311096 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:50.311681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:50.311953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:50.312217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:50.313895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:50.325698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:50.327092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:50.336257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1259/2486] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:58:54.646580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=55 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:58:57.117851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:57.119322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:57.136112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b94870-76e8-41bb-96e2-3ee13bc2cf61 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:57.140730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:57.141404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:57.141687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:57.141961 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] PATH_INFO: `/v3/users/c9efeb413ec245cfb83821f6b3433bbe` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:57.143107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9efeb413ec245cfb83821f6b3433bbe)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:57.145080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:57.196124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9efeb413ec245cfb83821f6b3433bbe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.196831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9efeb413ec245cfb83821f6b3433bbe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.204908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9efeb413ec245cfb83821f6b3433bbe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.222524 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9efeb413ec245cfb83821f6b3433bbe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.231424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9efeb413ec245cfb83821f6b3433bbe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.238726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1228/2487] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:58:57 2022] DELETE /identity/v3/users/c9efeb413ec245cfb83821f6b3433bbe => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:58:57.422228 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:57.437083 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8405f-2893-446b-81bf-6c4b1b440d27 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:57.503812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:57.504649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:57.505229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:57.505498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] PATH_INFO: `/v3/projects/550fb18df8234f6cb7d5fdf9770ce11e` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:57.506621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=550fb18df8234f6cb7d5fdf9770ce11e)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:57.508943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:57.592164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550fb18df8234f6cb7d5fdf9770ce11e', 'request_id': 'req-844cd586-ca73-4045-9902-719b468330aa'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.602821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550fb18df8234f6cb7d5fdf9770ce11e', 'request_id': 'req-844cd586-ca73-4045-9902-719b468330aa'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.609982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550fb18df8234f6cb7d5fdf9770ce11e', 'request_id': 'req-844cd586-ca73-4045-9902-719b468330aa'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.624498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] The token cache is being invalidate because project 550fb18df8234f6cb7d5fdf9770ce11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:58:57.624915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.625598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.627281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1260/2488] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:58:57 2022] DELETE /identity/v3/projects/550fb18df8234f6cb7d5fdf9770ce11e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:58:57.631444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:57.633235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:57.651309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa739a-855a-4edf-8d3d-2199fbcdc8b2 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:57.722691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:57.723322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:57.723598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:57.723863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] PATH_INFO: `/v3/users/c38f6f8c720045ce8200473b368fa79f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:57.724973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c38f6f8c720045ce8200473b368fa79f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:57.726899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:57.775102 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c38f6f8c720045ce8200473b368fa79f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.775765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c38f6f8c720045ce8200473b368fa79f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.785290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c38f6f8c720045ce8200473b368fa79f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.816411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c38f6f8c720045ce8200473b368fa79f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.818745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:58:57.824371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c38f6f8c720045ce8200473b368fa79f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:57.835419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1229/2489] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:58:57 2022] DELETE /identity/v3/users/c38f6f8c720045ce8200473b368fa79f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:58:58.002179 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:58:58.003616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:58:58.022050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844cd586-ca73-4045-9902-719b468330aa admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:58:58.034158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:58:58.080058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:58:58.080687 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:58:58.080966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:58:58.081272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] PATH_INFO: `/v3/projects/d4665cf1f3a74bde87f7eabf701f4d22` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:58:58.082375 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4665cf1f3a74bde87f7eabf701f4d22)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:58:58.084273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:58:58.129804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4665cf1f3a74bde87f7eabf701f4d22', 'request_id': 'req-a1f65abd-11b7-4c15-892f-5e5e51825c2a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:58.135806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4665cf1f3a74bde87f7eabf701f4d22', 'request_id': 'req-a1f65abd-11b7-4c15-892f-5e5e51825c2a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:58.141351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4665cf1f3a74bde87f7eabf701f4d22', 'request_id': 'req-a1f65abd-11b7-4c15-892f-5e5e51825c2a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:58.153720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] The token cache is being invalidate because project d4665cf1f3a74bde87f7eabf701f4d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:58:58.154005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:58.154617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1f65abd-11b7-4c15-892f-5e5e51825c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:58:58.156189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1261/2490] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:58:58 2022] DELETE /identity/v3/projects/d4665cf1f3a74bde87f7eabf701f4d22 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:07.837232 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:08.150167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:10.083411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:10.084799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:10.150528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d04620-1035-42eb-a7d5-3fed31ccbd3f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:10.203536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:10.204148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:10.204426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:10.204703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:10.206467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:10.269511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:10.270982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:10.280838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1230/2491] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:10.507801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b037ee3-7034-4d97-b2c3-f546132657a8 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:10.508591 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b037ee3-7034-4d97-b2c3-f546132657a8 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:10.508840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b037ee3-7034-4d97-b2c3-f546132657a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:10.562347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5b037ee3-7034-4d97-b2c3-f546132657a8 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:10.563572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5b037ee3-7034-4d97-b2c3-f546132657a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:10.631032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1262/2492] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:59:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:10.678112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:10.679580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:10.747536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:10.772755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f519c05-81c9-47e0-8e0a-ec8e367bdd43 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:10.779050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb8f27-1f37-47ed-8bb7-1eaf6225b142 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:10.779655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb8f27-1f37-47ed-8bb7-1eaf6225b142 service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:10.779933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb8f27-1f37-47ed-8bb7-1eaf6225b142 service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:10.780201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb8f27-1f37-47ed-8bb7-1eaf6225b142 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:10.788483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb8f27-1f37-47ed-8bb7-1eaf6225b142 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:10.790109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb8f27-1f37-47ed-8bb7-1eaf6225b142 service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:10.798939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1231/2493] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:59:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:59:14.242623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b037ee3-7034-4d97-b2c3-f546132657a8 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:14.244088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5b037ee3-7034-4d97-b2c3-f546132657a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:14.264558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b037ee3-7034-4d97-b2c3-f546132657a8 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:14.272190 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:14.273210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:14.273592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:14.274114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] PATH_INFO: `/v3/users/83f9dc8c41604f0e86eaf57202285cc4` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:14.291224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f9dc8c41604f0e86eaf57202285cc4)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:14.294254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:14.354493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f9dc8c41604f0e86eaf57202285cc4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.355747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f9dc8c41604f0e86eaf57202285cc4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.370057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f9dc8c41604f0e86eaf57202285cc4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.379900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f9dc8c41604f0e86eaf57202285cc4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.380721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-25dc646d-37a2-4ab3-bc30-c25b107a6754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f9dc8c41604f0e86eaf57202285cc4'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.395260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1263/2494] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:59:14 2022] DELETE /identity/v3/users/83f9dc8c41604f0e86eaf57202285cc4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:14.577981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb8f27-1f37-47ed-8bb7-1eaf6225b142 service swift] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:14.598597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb8f27-1f37-47ed-8bb7-1eaf6225b142 service swift] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:14.682636 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:14.683307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:14.683646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:14.684109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] PATH_INFO: `/v3/projects/1ed13a0f61d34a80898093dc1d8e6610` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:14.696362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ed13a0f61d34a80898093dc1d8e6610)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:14.698456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:14.753823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ed13a0f61d34a80898093dc1d8e6610', 'request_id': 'req-d8ff303e-65e3-48aa-97d9-37f704897479'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.760906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ed13a0f61d34a80898093dc1d8e6610', 'request_id': 'req-d8ff303e-65e3-48aa-97d9-37f704897479'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.768608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ed13a0f61d34a80898093dc1d8e6610', 'request_id': 'req-d8ff303e-65e3-48aa-97d9-37f704897479'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.784820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] The token cache is being invalidate because project 1ed13a0f61d34a80898093dc1d8e6610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:59:14.785198 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.785829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d8ff303e-65e3-48aa-97d9-37f704897479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:14.787456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1232/2495] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:59:14 2022] DELETE /identity/v3/projects/1ed13a0f61d34a80898093dc1d8e6610 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:14.793675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5410-ce7d-49a6-832e-821e22ad2503 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:14.793961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5410-ce7d-49a6-832e-821e22ad2503 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:14.794235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5410-ce7d-49a6-832e-821e22ad2503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:14.818271 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f41d5410-ce7d-49a6-832e-821e22ad2503 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:14.819761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f41d5410-ce7d-49a6-832e-821e22ad2503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:14.862816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:14.912658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1264/2496] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:59:14.917337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2aba9-1950-4438-a9ed-09ddcf74a68e None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:14.918251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2aba9-1950-4438-a9ed-09ddcf74a68e None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:14.918597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2aba9-1950-4438-a9ed-09ddcf74a68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:14.946939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-78d2aba9-1950-4438-a9ed-09ddcf74a68e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:14.948829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-78d2aba9-1950-4438-a9ed-09ddcf74a68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:14.961992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1233/2497] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:14.963353 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d5410-ce7d-49a6-832e-821e22ad2503 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:14.977978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d5410-ce7d-49a6-832e-821e22ad2503 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:14.997642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:14.998775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:14.999288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:14.999787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.000863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:59:15.001453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:15.009450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:15.040154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1265/2498] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:59:15.047770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2aba9-1950-4438-a9ed-09ddcf74a68e None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:15.057452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2aba9-1950-4438-a9ed-09ddcf74a68e None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:15.063507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d99c87-e3e7-40cb-b70f-7a94a0559902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:15.064244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d99c87-e3e7-40cb-b70f-7a94a0559902 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:15.064616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d99c87-e3e7-40cb-b70f-7a94a0559902 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:15.064982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d99c87-e3e7-40cb-b70f-7a94a0559902 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.065746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d99c87-e3e7-40cb-b70f-7a94a0559902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:15.067736 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d99c87-e3e7-40cb-b70f-7a94a0559902 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:15.110980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1234/2499] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:15 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:59:15.112239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:15.123347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a665a-3ff6-42dd-b0f5-75016c70a06d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:15.187925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c37e3-f388-4ee9-bdaa-91a8db710cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:15.188950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c37e3-f388-4ee9-bdaa-91a8db710cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:15.189348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c37e3-f388-4ee9-bdaa-91a8db710cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:15.189549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c37e3-f388-4ee9-bdaa-91a8db710cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.190175 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c37e3-f388-4ee9-bdaa-91a8db710cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:15.192084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c37e3-f388-4ee9-bdaa-91a8db710cb3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:15.217654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1266/2500] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:15 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:15.221784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d99c87-e3e7-40cb-b70f-7a94a0559902 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:15.232947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d99c87-e3e7-40cb-b70f-7a94a0559902 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:15.237935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a88806-cbd9-464f-8617-3e0993c9204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:15.238624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a88806-cbd9-464f-8617-3e0993c9204b admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:15.238967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a88806-cbd9-464f-8617-3e0993c9204b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:15.239309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a88806-cbd9-464f-8617-3e0993c9204b admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.240095 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a88806-cbd9-464f-8617-3e0993c9204b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:15.242077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a88806-cbd9-464f-8617-3e0993c9204b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:15.255825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1235/2501] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:15.257301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c37e3-f388-4ee9-bdaa-91a8db710cb3 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:15.269991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c37e3-f388-4ee9-bdaa-91a8db710cb3 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:15.274953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b114ab0-cc45-4add-84bd-de4468feedda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:15.275684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b114ab0-cc45-4add-84bd-de4468feedda admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:15.276063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b114ab0-cc45-4add-84bd-de4468feedda admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:15.276428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b114ab0-cc45-4add-84bd-de4468feedda admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.277331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b114ab0-cc45-4add-84bd-de4468feedda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:15.279016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b114ab0-cc45-4add-84bd-de4468feedda admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:15.292740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1267/2502] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:15.294544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a88806-cbd9-464f-8617-3e0993c9204b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:15.305959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a88806-cbd9-464f-8617-3e0993c9204b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:15.311661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:15.312444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:15.312845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:15.313259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] PATH_INFO: `/v3/projects/d2acf1bb82b24fb6b170b994658c8c9e/users/f7613fad38a3428492dae4f987c32447/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.319112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:15.336011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2acf1bb82b24fb6b170b994658c8c9e, user_id=f7613fad38a3428492dae4f987c32447, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:15.341110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:15.359069 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1236/2503] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:15 2022] PUT /identity/v3/projects/d2acf1bb82b24fb6b170b994658c8c9e/users/f7613fad38a3428492dae4f987c32447/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:15.375410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269653e0-3bac-420a-9ad2-b30e4509f612 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:15.375767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269653e0-3bac-420a-9ad2-b30e4509f612 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:15.376088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269653e0-3bac-420a-9ad2-b30e4509f612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.417216 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-269653e0-3bac-420a-9ad2-b30e4509f612 None None] MFA Rules not processed for user `f7613fad38a3428492dae4f987c32447`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:15.418539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-269653e0-3bac-420a-9ad2-b30e4509f612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:15.513193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1268/2504] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:15.524184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:15.525986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:15.607847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653044a8-d8e6-4aa5-8965-7221d751c0dd admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:15.614669 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3b7ff-b059-4227-b405-4e0b8a91ec13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:15.615715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3b7ff-b059-4227-b405-4e0b8a91ec13 service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:15.616096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3b7ff-b059-4227-b405-4e0b8a91ec13 service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:15.616468 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3b7ff-b059-4227-b405-4e0b8a91ec13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.624836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3b7ff-b059-4227-b405-4e0b8a91ec13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:15.626571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3b7ff-b059-4227-b405-4e0b8a91ec13 service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:15.637850 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1237/2505] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:15.870072 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269653e0-3bac-420a-9ad2-b30e4509f612 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:15.905341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:15.951381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269653e0-3bac-420a-9ad2-b30e4509f612 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:15.956006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:15.956639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:15.956917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:15.957234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] PATH_INFO: `/v3/users/f7613fad38a3428492dae4f987c32447` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:15.958528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7613fad38a3428492dae4f987c32447)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:15.960437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:16.018036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7613fad38a3428492dae4f987c32447'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.018655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7613fad38a3428492dae4f987c32447'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.032949 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7613fad38a3428492dae4f987c32447'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.046635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7613fad38a3428492dae4f987c32447'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.046635 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7613fad38a3428492dae4f987c32447'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.065607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1269/2506] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:59:15 2022] DELETE /identity/v3/users/f7613fad38a3428492dae4f987c32447 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:16.072855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c66c5f-ed63-41f0-9e02-8b11e7b58617 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:16.073193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c66c5f-ed63-41f0-9e02-8b11e7b58617 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:16.073476 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c66c5f-ed63-41f0-9e02-8b11e7b58617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:16.103027 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-13c66c5f-ed63-41f0-9e02-8b11e7b58617 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:16.105855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-13c66c5f-ed63-41f0-9e02-8b11e7b58617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:16.193765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1238/2507] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:16.205495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:16.207208 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:16.285161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a089a-3942-4da2-8663-e5e096d0c840 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:16.288863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09753d3b-dc11-45dd-a478-0e900c454d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:16.289507 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753d3b-dc11-45dd-a478-0e900c454d66 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:16.289785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753d3b-dc11-45dd-a478-0e900c454d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:16.290059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753d3b-dc11-45dd-a478-0e900c454d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:16.301212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09753d3b-dc11-45dd-a478-0e900c454d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:16.302663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09753d3b-dc11-45dd-a478-0e900c454d66 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:16.312898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1270/2508] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:16.652051 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c66c5f-ed63-41f0-9e02-8b11e7b58617 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:16.654207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-13c66c5f-ed63-41f0-9e02-8b11e7b58617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:16.676909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c66c5f-ed63-41f0-9e02-8b11e7b58617 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:16.689490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:16.690123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:16.690397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:16.690674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] PATH_INFO: `/v3/projects/d2acf1bb82b24fb6b170b994658c8c9e` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:16.693650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2acf1bb82b24fb6b170b994658c8c9e)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:16.695551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:16.748048 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2acf1bb82b24fb6b170b994658c8c9e', 'request_id': 'req-e7580894-0e67-4c9c-9e00-35b80dd796f6'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.755069 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2acf1bb82b24fb6b170b994658c8c9e', 'request_id': 'req-e7580894-0e67-4c9c-9e00-35b80dd796f6'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.762653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2acf1bb82b24fb6b170b994658c8c9e', 'request_id': 'req-e7580894-0e67-4c9c-9e00-35b80dd796f6'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.765473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:16.778812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] The token cache is being invalidate because project d2acf1bb82b24fb6b170b994658c8c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:59:16.779262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.780612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e7580894-0e67-4c9c-9e00-35b80dd796f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:16.782740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1239/2509] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:59:16 2022] DELETE /identity/v3/projects/d2acf1bb82b24fb6b170b994658c8c9e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:16.787637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadab803-6cb8-4c7e-83f1-0b21d38bb120 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:16.787918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadab803-6cb8-4c7e-83f1-0b21d38bb120 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:16.788211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadab803-6cb8-4c7e-83f1-0b21d38bb120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:16.810236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-dadab803-6cb8-4c7e-83f1-0b21d38bb120 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:16.811574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-dadab803-6cb8-4c7e-83f1-0b21d38bb120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:16.864935 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:16.888229 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1271/2510] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:16.901818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fbc7e-0584-4872-ae69-0c647933c54b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:16.901818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fbc7e-0584-4872-ae69-0c647933c54b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:16.901818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fbc7e-0584-4872-ae69-0c647933c54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:16.925195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-741fbc7e-0584-4872-ae69-0c647933c54b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:16.925195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-741fbc7e-0584-4872-ae69-0c647933c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:16.937339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1240/2511] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:16.938381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadab803-6cb8-4c7e-83f1-0b21d38bb120 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:16.949640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadab803-6cb8-4c7e-83f1-0b21d38bb120 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:16.954572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:16.955203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:16.955480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:16.955751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:16.956410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:59:16.956713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:16.958366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:16.971141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1272/2512] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 01:59:16.975829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fbc7e-0584-4872-ae69-0c647933c54b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:16.996239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fbc7e-0584-4872-ae69-0c647933c54b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.001339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac73af1-d65d-441b-94ea-739d945dd021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.001980 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac73af1-d65d-441b-94ea-739d945dd021 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.002258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac73af1-d65d-441b-94ea-739d945dd021 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.002590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac73af1-d65d-441b-94ea-739d945dd021 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.003243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac73af1-d65d-441b-94ea-739d945dd021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.005367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac73af1-d65d-441b-94ea-739d945dd021 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.035801 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1241/2513] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:16 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:17.042267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.057094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350c499-233f-4376-adf7-086bbbd0cade admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.123899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142692a3-3139-48b1-8b98-b1e473b29993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.124456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142692a3-3139-48b1-8b98-b1e473b29993 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.124728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142692a3-3139-48b1-8b98-b1e473b29993 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.124993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142692a3-3139-48b1-8b98-b1e473b29993 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.125679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142692a3-3139-48b1-8b98-b1e473b29993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.127570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142692a3-3139-48b1-8b98-b1e473b29993 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.149247 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1273/2514] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:17 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:17.154346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac73af1-d65d-441b-94ea-739d945dd021 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.165665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac73af1-d65d-441b-94ea-739d945dd021 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.173825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4d95-4523-424c-bb9e-8068e92a4091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.174444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4d95-4523-424c-bb9e-8068e92a4091 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.174857 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4d95-4523-424c-bb9e-8068e92a4091 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.175136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4d95-4523-424c-bb9e-8068e92a4091 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.175783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd4d95-4523-424c-bb9e-8068e92a4091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.177439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd4d95-4523-424c-bb9e-8068e92a4091 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.189051 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1242/2515] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:17.192861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142692a3-3139-48b1-8b98-b1e473b29993 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.204176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142692a3-3139-48b1-8b98-b1e473b29993 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.210107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad51cd-9d8c-4a4e-b85e-914ac567f4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.211076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad51cd-9d8c-4a4e-b85e-914ac567f4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.211560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad51cd-9d8c-4a4e-b85e-914ac567f4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.211999 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad51cd-9d8c-4a4e-b85e-914ac567f4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.212792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad51cd-9d8c-4a4e-b85e-914ac567f4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.214528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad51cd-9d8c-4a4e-b85e-914ac567f4b3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.226743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1274/2516] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:17.231417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4d95-4523-424c-bb9e-8068e92a4091 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.242950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4d95-4523-424c-bb9e-8068e92a4091 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.248691 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cc652-44a2-4630-baf5-f0dd5b502990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.249678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc652-44a2-4630-baf5-f0dd5b502990 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.250055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc652-44a2-4630-baf5-f0dd5b502990 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.250494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc652-44a2-4630-baf5-f0dd5b502990 admin admin] PATH_INFO: `/v3/projects/c90f48548d3f47d58fe49931d113750c/users/a781a7a48bc648be927febabfe4b6c39/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.266924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc652-44a2-4630-baf5-f0dd5b502990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90f48548d3f47d58fe49931d113750c, user_id=a781a7a48bc648be927febabfe4b6c39, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.269882 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc652-44a2-4630-baf5-f0dd5b502990 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.299622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1243/2517] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:17 2022] PUT /identity/v3/projects/c90f48548d3f47d58fe49931d113750c/users/a781a7a48bc648be927febabfe4b6c39/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:59:17.301845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd552e9-8edf-4cc4-9867-6c60cad6a1f2 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.302366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd552e9-8edf-4cc4-9867-6c60cad6a1f2 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.302366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd552e9-8edf-4cc4-9867-6c60cad6a1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.342817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-dbd552e9-8edf-4cc4-9867-6c60cad6a1f2 None None] MFA Rules not processed for user `a781a7a48bc648be927febabfe4b6c39`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:17.344337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-dbd552e9-8edf-4cc4-9867-6c60cad6a1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:17.445163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1275/2518] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:17.450320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cc652-44a2-4630-baf5-f0dd5b502990 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.559183 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cc652-44a2-4630-baf5-f0dd5b502990 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.565813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c516b3f-ccfb-4a43-b653-64988c019f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.566391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c516b3f-ccfb-4a43-b653-64988c019f93 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.566671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c516b3f-ccfb-4a43-b653-64988c019f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.566943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c516b3f-ccfb-4a43-b653-64988c019f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.567573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c516b3f-ccfb-4a43-b653-64988c019f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.569810 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c516b3f-ccfb-4a43-b653-64988c019f93 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.602207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1244/2519] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:17 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:17.606854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd552e9-8edf-4cc4-9867-6c60cad6a1f2 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.621472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd552e9-8edf-4cc4-9867-6c60cad6a1f2 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.674272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:17.696256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d374df-7385-42e0-be4b-599c5d702096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.697240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d374df-7385-42e0-be4b-599c5d702096 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.697861 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d374df-7385-42e0-be4b-599c5d702096 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.697987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d374df-7385-42e0-be4b-599c5d702096 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.699443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d374df-7385-42e0-be4b-599c5d702096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.701649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d374df-7385-42e0-be4b-599c5d702096 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.733259 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1276/2520] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:17 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:17.734903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c516b3f-ccfb-4a43-b653-64988c019f93 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.748344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c516b3f-ccfb-4a43-b653-64988c019f93 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.755382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14566758-f69a-478f-9f1f-e2aa8be036fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.756011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14566758-f69a-478f-9f1f-e2aa8be036fd admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.756301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14566758-f69a-478f-9f1f-e2aa8be036fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.756586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14566758-f69a-478f-9f1f-e2aa8be036fd admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.757240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14566758-f69a-478f-9f1f-e2aa8be036fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.758879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14566758-f69a-478f-9f1f-e2aa8be036fd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.773661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1245/2521] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:17.778587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d374df-7385-42e0-be4b-599c5d702096 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.790634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d374df-7385-42e0-be4b-599c5d702096 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.796104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8454b7-4841-4901-936d-6103fd446b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.796755 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8454b7-4841-4901-936d-6103fd446b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.797029 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8454b7-4841-4901-936d-6103fd446b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.797377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8454b7-4841-4901-936d-6103fd446b13 admin admin] PATH_INFO: `/v3/projects/60f2bf3ae586489f8661ef5d582c9401/users/abaf7735a2e84501b0af26d627e33dfe/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.821189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8454b7-4841-4901-936d-6103fd446b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f2bf3ae586489f8661ef5d582c9401, user_id=abaf7735a2e84501b0af26d627e33dfe, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.825156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8454b7-4841-4901-936d-6103fd446b13 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.840540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1277/2522] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:17 2022] PUT /identity/v3/projects/60f2bf3ae586489f8661ef5d582c9401/users/abaf7735a2e84501b0af26d627e33dfe/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:59:17.844610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14566758-f69a-478f-9f1f-e2aa8be036fd admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.857864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14566758-f69a-478f-9f1f-e2aa8be036fd admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.936800 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c53cfb-bbef-49fb-993a-26139d412891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.938769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c53cfb-bbef-49fb-993a-26139d412891 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.939604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c53cfb-bbef-49fb-993a-26139d412891 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.942624 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c53cfb-bbef-49fb-993a-26139d412891 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.944101 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c53cfb-bbef-49fb-993a-26139d412891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.949262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c53cfb-bbef-49fb-993a-26139d412891 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:17.955318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:17.962304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1246/2523] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:17 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:17.966460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8454b7-4841-4901-936d-6103fd446b13 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:17.979558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8454b7-4841-4901-936d-6103fd446b13 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:17.984585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:17.985532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:17.985854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:17.986138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] PATH_INFO: `/v3/projects/60f2bf3ae586489f8661ef5d582c9401/users/abaf7735a2e84501b0af26d627e33dfe/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:17.988076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f2bf3ae586489f8661ef5d582c9401, user_id=abaf7735a2e84501b0af26d627e33dfe, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:17.991390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:18.019626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1278/2524] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:17 2022] PUT /identity/v3/projects/60f2bf3ae586489f8661ef5d582c9401/users/abaf7735a2e84501b0af26d627e33dfe/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:18.023874 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f66de2-847c-4866-8568-5164db8af2ce None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:18.024333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f66de2-847c-4866-8568-5164db8af2ce None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:18.024613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f66de2-847c-4866-8568-5164db8af2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:18.068607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e8f66de2-847c-4866-8568-5164db8af2ce None None] MFA Rules not processed for user `abaf7735a2e84501b0af26d627e33dfe`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:18.069868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e8f66de2-847c-4866-8568-5164db8af2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:18.171208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1247/2525] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:18.179750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:18.181439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:18.254879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad50af-47db-452f-bf89-ae1eeae11f87 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:18.259139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f728a53-503f-4e2c-9be6-0fcceb38c297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:18.259826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f728a53-503f-4e2c-9be6-0fcceb38c297 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:18.260172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f728a53-503f-4e2c-9be6-0fcceb38c297 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:18.260511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f728a53-503f-4e2c-9be6-0fcceb38c297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:18.271236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f728a53-503f-4e2c-9be6-0fcceb38c297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:18.272776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f728a53-503f-4e2c-9be6-0fcceb38c297 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:18.283374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1279/2526] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:18.858310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f66de2-847c-4866-8568-5164db8af2ce None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:18.943862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f66de2-847c-4866-8568-5164db8af2ce None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:18.952422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:18.953584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:18.953926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:18.954406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] PATH_INFO: `/v3/users/a781a7a48bc648be927febabfe4b6c39` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:18.955538 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a781a7a48bc648be927febabfe4b6c39)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:18.957710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:19.014651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a781a7a48bc648be927febabfe4b6c39'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.015732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a781a7a48bc648be927febabfe4b6c39'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.025496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a781a7a48bc648be927febabfe4b6c39'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.042787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a781a7a48bc648be927febabfe4b6c39'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.052667 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a781a7a48bc648be927febabfe4b6c39'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.061143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1248/2527] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:59:18 2022] DELETE /identity/v3/users/a781a7a48bc648be927febabfe4b6c39 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:19.064299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2200137-bada-4f4e-a9b9-d7d97be22bdb None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:19.064764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2200137-bada-4f4e-a9b9-d7d97be22bdb None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:19.064874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2200137-bada-4f4e-a9b9-d7d97be22bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:19.087232 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b2200137-bada-4f4e-a9b9-d7d97be22bdb None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:19.088616 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b2200137-bada-4f4e-a9b9-d7d97be22bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:19.161127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1280/2528] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:19.171053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:19.172496 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:19.179879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:19.243974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26dd05c-e756-41f8-acdf-0060fd844959 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:19.248511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b7649-ea3f-400a-a676-4cc6707c63b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:19.249148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b7649-ea3f-400a-a676-4cc6707c63b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:19.249432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b7649-ea3f-400a-a676-4cc6707c63b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:19.249710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b7649-ea3f-400a-a676-4cc6707c63b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:19.263426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b7649-ea3f-400a-a676-4cc6707c63b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:19.264855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b7649-ea3f-400a-a676-4cc6707c63b6 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:19.275083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1249/2529] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:19.588944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2200137-bada-4f4e-a9b9-d7d97be22bdb None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:19.590964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b2200137-bada-4f4e-a9b9-d7d97be22bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:19.627036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2200137-bada-4f4e-a9b9-d7d97be22bdb None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:19.633293 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:19.633936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:19.634209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:19.634546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] PATH_INFO: `/v3/projects/c90f48548d3f47d58fe49931d113750c` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:19.635675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90f48548d3f47d58fe49931d113750c)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:19.638263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:19.692710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:19.704086 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90f48548d3f47d58fe49931d113750c', 'request_id': 'req-9e10de67-ed0d-47fb-8883-45ad073476bf'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.711437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90f48548d3f47d58fe49931d113750c', 'request_id': 'req-9e10de67-ed0d-47fb-8883-45ad073476bf'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.721301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90f48548d3f47d58fe49931d113750c', 'request_id': 'req-9e10de67-ed0d-47fb-8883-45ad073476bf'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.738237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] The token cache is being invalidate because project c90f48548d3f47d58fe49931d113750c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:59:19.738567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.739207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.741269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1281/2530] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:59:19 2022] DELETE /identity/v3/projects/c90f48548d3f47d58fe49931d113750c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:19.748860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b7649-ea3f-400a-a676-4cc6707c63b6 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:19.762956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b7649-ea3f-400a-a676-4cc6707c63b6 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:19.844098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:19.844747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:19.845106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:19.845397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] PATH_INFO: `/v3/users/abaf7735a2e84501b0af26d627e33dfe` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:19.846600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abaf7735a2e84501b0af26d627e33dfe)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:19.848578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:19.918784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abaf7735a2e84501b0af26d627e33dfe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.919792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abaf7735a2e84501b0af26d627e33dfe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.928777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abaf7735a2e84501b0af26d627e33dfe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.942402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abaf7735a2e84501b0af26d627e33dfe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.950252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0213d6c9-eccb-4451-9f7d-730ffec539b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abaf7735a2e84501b0af26d627e33dfe'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:19.960405 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1250/2531] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:59:19 2022] DELETE /identity/v3/users/abaf7735a2e84501b0af26d627e33dfe => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:20.297712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:20.299192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:20.317903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10de67-ed0d-47fb-8883-45ad073476bf admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:20.392384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:20.392384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.392384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.392840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] PATH_INFO: `/v3/projects/60f2bf3ae586489f8661ef5d582c9401` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.394442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f2bf3ae586489f8661ef5d582c9401)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:20.396044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:20.451825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f2bf3ae586489f8661ef5d582c9401', 'request_id': 'req-a0473bfc-e294-4555-8921-9531a4cfdf0d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:20.458248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f2bf3ae586489f8661ef5d582c9401', 'request_id': 'req-a0473bfc-e294-4555-8921-9531a4cfdf0d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:20.464528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f2bf3ae586489f8661ef5d582c9401', 'request_id': 'req-a0473bfc-e294-4555-8921-9531a4cfdf0d'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:20.479558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] The token cache is being invalidate because project 60f2bf3ae586489f8661ef5d582c9401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:59:20.479950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:20.480650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a0473bfc-e294-4555-8921-9531a4cfdf0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:20.482407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1282/2532] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:59:20 2022] DELETE /identity/v3/projects/60f2bf3ae586489f8661ef5d582c9401 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:20.487049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56034ac-7263-4037-8cd2-f150cb627932 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.487333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56034ac-7263-4037-8cd2-f150cb627932 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.487604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56034ac-7263-4037-8cd2-f150cb627932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.510083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-c56034ac-7263-4037-8cd2-f150cb627932 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:20.511945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c56034ac-7263-4037-8cd2-f150cb627932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:20.536485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:20.587131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1251/2533] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:20.597248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9308c9-ec1f-4c9f-ab17-4533d3389628 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.597721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9308c9-ec1f-4c9f-ab17-4533d3389628 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.598126 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9308c9-ec1f-4c9f-ab17-4533d3389628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.620886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-eb9308c9-ec1f-4c9f-ab17-4533d3389628 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:20.622263 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-eb9308c9-ec1f-4c9f-ab17-4533d3389628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:20.630827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1283/2534] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:59:20.635201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56034ac-7263-4037-8cd2-f150cb627932 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:20.645590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56034ac-7263-4037-8cd2-f150cb627932 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:20.650467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:20.651075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.651352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.652250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.652901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:59:20.653256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:20.655147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:20.671547 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1252/2535] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:59:20.672632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9308c9-ec1f-4c9f-ab17-4533d3389628 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:20.682111 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9308c9-ec1f-4c9f-ab17-4533d3389628 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:20.686846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a75bd8-4179-4107-997d-501c261cf415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:20.687444 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a75bd8-4179-4107-997d-501c261cf415 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.687724 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a75bd8-4179-4107-997d-501c261cf415 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.687995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a75bd8-4179-4107-997d-501c261cf415 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.690497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a75bd8-4179-4107-997d-501c261cf415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:20.692425 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a75bd8-4179-4107-997d-501c261cf415 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:20.721446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1284/2536] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:20 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:20.725989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:20.736321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325ae99-d292-492f-8a2c-5414c222dc97 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:20.794639 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aa24e-1a2b-4547-9525-898cfcd6152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:20.795267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aa24e-1a2b-4547-9525-898cfcd6152a admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.795541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aa24e-1a2b-4547-9525-898cfcd6152a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.795809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aa24e-1a2b-4547-9525-898cfcd6152a admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.796434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540aa24e-1a2b-4547-9525-898cfcd6152a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:20.798337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540aa24e-1a2b-4547-9525-898cfcd6152a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:20.819370 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1253/2537] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:20 2022] POST /identity/v3/users => generated 279 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:59:20.823961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a75bd8-4179-4107-997d-501c261cf415 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:20.834057 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a75bd8-4179-4107-997d-501c261cf415 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:20.838620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34217b99-72e7-4790-b9c9-ae56d39387c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:20.839763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34217b99-72e7-4790-b9c9-ae56d39387c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.840047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34217b99-72e7-4790-b9c9-ae56d39387c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.840317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34217b99-72e7-4790-b9c9-ae56d39387c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.840947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34217b99-72e7-4790-b9c9-ae56d39387c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:20.842556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34217b99-72e7-4790-b9c9-ae56d39387c8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:20.852252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1285/2538] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:20.856396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aa24e-1a2b-4547-9525-898cfcd6152a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:20.866771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aa24e-1a2b-4547-9525-898cfcd6152a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:20.871357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1b091-d1f3-40a8-b99e-96c3cc67c7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:20.871946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1b091-d1f3-40a8-b99e-96c3cc67c7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.872221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1b091-d1f3-40a8-b99e-96c3cc67c7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.872487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1b091-d1f3-40a8-b99e-96c3cc67c7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.873149 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1b091-d1f3-40a8-b99e-96c3cc67c7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:20.874715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1b091-d1f3-40a8-b99e-96c3cc67c7b6 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:20.884999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1254/2539] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:20.889418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34217b99-72e7-4790-b9c9-ae56d39387c8 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:20.899276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34217b99-72e7-4790-b9c9-ae56d39387c8 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:20.903957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6281-545f-4dc2-8602-8b6432114d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:20.904624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6281-545f-4dc2-8602-8b6432114d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.904901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6281-545f-4dc2-8602-8b6432114d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.905213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a6281-545f-4dc2-8602-8b6432114d20 admin admin] PATH_INFO: `/v3/projects/ab6a05d721c24383a92e3d239e782c2a/users/91cf485ce0db45ba9597d28aa8768350/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.917091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:20.921162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6281-545f-4dc2-8602-8b6432114d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6a05d721c24383a92e3d239e782c2a, user_id=91cf485ce0db45ba9597d28aa8768350, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:20.924758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a6281-545f-4dc2-8602-8b6432114d20 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:20.938524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1286/2540] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:20 2022] PUT /identity/v3/projects/ab6a05d721c24383a92e3d239e782c2a/users/91cf485ce0db45ba9597d28aa8768350/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:20.948634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85165c9-1e25-492c-a063-346409d91117 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:20.949103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85165c9-1e25-492c-a063-346409d91117 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:20.949213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85165c9-1e25-492c-a063-346409d91117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:20.975731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b85165c9-1e25-492c-a063-346409d91117 None None] MFA Rules not processed for user `91cf485ce0db45ba9597d28aa8768350`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:20.976913 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b85165c9-1e25-492c-a063-346409d91117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:21.044924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1255/2541] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:21.049493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6281-545f-4dc2-8602-8b6432114d20 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.112450 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a6281-545f-4dc2-8602-8b6432114d20 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.117477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e970c4-d8fb-4e15-bcaa-28d3869b14d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.118080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e970c4-d8fb-4e15-bcaa-28d3869b14d8 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.118358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e970c4-d8fb-4e15-bcaa-28d3869b14d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.118632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e970c4-d8fb-4e15-bcaa-28d3869b14d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.119644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e970c4-d8fb-4e15-bcaa-28d3869b14d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.121268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e970c4-d8fb-4e15-bcaa-28d3869b14d8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.148893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1287/2542] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:21 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 01:59:21.153197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85165c9-1e25-492c-a063-346409d91117 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.164862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85165c9-1e25-492c-a063-346409d91117 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.184610 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:21.224457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab9c3f-deea-4f56-b956-43fc10f13ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.225053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9c3f-deea-4f56-b956-43fc10f13ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.225362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9c3f-deea-4f56-b956-43fc10f13ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.225629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab9c3f-deea-4f56-b956-43fc10f13ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.226238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab9c3f-deea-4f56-b956-43fc10f13ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.228124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab9c3f-deea-4f56-b956-43fc10f13ce6 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.252008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1256/2543] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:21 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:21.253194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e970c4-d8fb-4e15-bcaa-28d3869b14d8 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.263525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e970c4-d8fb-4e15-bcaa-28d3869b14d8 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.268139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2a7f2-eac0-4510-b086-7868f56a3b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.268729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a7f2-eac0-4510-b086-7868f56a3b22 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.269034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a7f2-eac0-4510-b086-7868f56a3b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.269330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a7f2-eac0-4510-b086-7868f56a3b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.269962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2a7f2-eac0-4510-b086-7868f56a3b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.271547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2a7f2-eac0-4510-b086-7868f56a3b22 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.282405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1288/2544] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 01:59:21.286616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab9c3f-deea-4f56-b956-43fc10f13ce6 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.297226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab9c3f-deea-4f56-b956-43fc10f13ce6 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.301822 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4663545-2407-46f8-b693-26d3c4e537ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.302461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4663545-2407-46f8-b693-26d3c4e537ad admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.302738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4663545-2407-46f8-b693-26d3c4e537ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.303006 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4663545-2407-46f8-b693-26d3c4e537ad admin admin] PATH_INFO: `/v3/projects/820422e35dab4506adc06c761c7339c0/users/b9eddbef315245d29cc399798a87616e/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.317307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4663545-2407-46f8-b693-26d3c4e537ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=820422e35dab4506adc06c761c7339c0, user_id=b9eddbef315245d29cc399798a87616e, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.319963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4663545-2407-46f8-b693-26d3c4e537ad admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.333103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1257/2545] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:21 2022] PUT /identity/v3/projects/820422e35dab4506adc06c761c7339c0/users/b9eddbef315245d29cc399798a87616e/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:21.337022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2a7f2-eac0-4510-b086-7868f56a3b22 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.348442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2a7f2-eac0-4510-b086-7868f56a3b22 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.405119 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0417b4-00b8-4aab-914c-b78a148c701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.405647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0417b4-00b8-4aab-914c-b78a148c701d admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.405923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0417b4-00b8-4aab-914c-b78a148c701d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.406191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0417b4-00b8-4aab-914c-b78a148c701d admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.406820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0417b4-00b8-4aab-914c-b78a148c701d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.408386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0417b4-00b8-4aab-914c-b78a148c701d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.418201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1289/2546] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:21.422341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4663545-2407-46f8-b693-26d3c4e537ad admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.432536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4663545-2407-46f8-b693-26d3c4e537ad admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.437963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.438612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.438912 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.439169 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] PATH_INFO: `/v3/projects/820422e35dab4506adc06c761c7339c0/users/b9eddbef315245d29cc399798a87616e/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.443260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] RBAC: Authorizing `identity:create_grant(project_id=820422e35dab4506adc06c761c7339c0, user_id=b9eddbef315245d29cc399798a87616e, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.446343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.457778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1258/2547] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:21 2022] PUT /identity/v3/projects/820422e35dab4506adc06c761c7339c0/users/b9eddbef315245d29cc399798a87616e/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:21.467915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37432bd-81e5-4a29-92ba-fe57d5370d36 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.468195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37432bd-81e5-4a29-92ba-fe57d5370d36 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.468463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37432bd-81e5-4a29-92ba-fe57d5370d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.496203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-a37432bd-81e5-4a29-92ba-fe57d5370d36 None None] MFA Rules not processed for user `b9eddbef315245d29cc399798a87616e`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:21.497424 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a37432bd-81e5-4a29-92ba-fe57d5370d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:21.543506 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:21.566428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1290/2548] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:21.578413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.580419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:21.642219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d1b61-dd46-4040-8e66-29d7eda9708e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.645864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.646508 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.646785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.647054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.656018 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.657426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.666621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1259/2549] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:21.739433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37432bd-81e5-4a29-92ba-fe57d5370d36 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.749518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37432bd-81e5-4a29-92ba-fe57d5370d36 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.753170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7f4b8-25c2-4a51-842f-9a0b537a8fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.753769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7f4b8-25c2-4a51-842f-9a0b537a8fce service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.754045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7f4b8-25c2-4a51-842f-9a0b537a8fce service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.754318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7f4b8-25c2-4a51-842f-9a0b537a8fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.782317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:21.783833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:21.813471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7f4b8-25c2-4a51-842f-9a0b537a8fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.814902 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7f4b8-25c2-4a51-842f-9a0b537a8fce service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.824992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1291/2550] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:21.842100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4761138a-d31f-43bf-b3fb-48d5d06af1a8 service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:21.846017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac1d61-5d5e-4afa-90a6-fdc0f70a51ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:21.846729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac1d61-5d5e-4afa-90a6-fdc0f70a51ba service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:21.847084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac1d61-5d5e-4afa-90a6-fdc0f70a51ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:21.847429 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac1d61-5d5e-4afa-90a6-fdc0f70a51ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:21.849328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-31ac1d61-5d5e-4afa-90a6-fdc0f70a51ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:21.859577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac1d61-5d5e-4afa-90a6-fdc0f70a51ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:21.861085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac1d61-5d5e-4afa-90a6-fdc0f70a51ba service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:21.869816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1260/2551] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 01:59:25.400879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810175bb-fa54-4509-98dc-63204d13c96d None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:25.402059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810175bb-fa54-4509-98dc-63204d13c96d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:25.402213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810175bb-fa54-4509-98dc-63204d13c96d None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:25.403608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1292/2552] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:59:25.410042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c744087-fa04-4659-b129-5203a72af7c1 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:25.410526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c744087-fa04-4659-b129-5203a72af7c1 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:25.410631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c744087-fa04-4659-b129-5203a72af7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:25.446087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:25.449948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7c744087-fa04-4659-b129-5203a72af7c1 None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:25.451753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7c744087-fa04-4659-b129-5203a72af7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:25.522513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1261/2553] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:59:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:25.534878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810175bb-fa54-4509-98dc-63204d13c96d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:25.536699 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-810175bb-fa54-4509-98dc-63204d13c96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:25.617595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810175bb-fa54-4509-98dc-63204d13c96d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:25.621840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381cac1-3173-4403-ac2f-adba24315028 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029447e782484e8f8a704b683ede1819', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:25.622449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381cac1-3173-4403-ac2f-adba24315028 service glance] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:25.622729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381cac1-3173-4403-ac2f-adba24315028 service glance] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:25.623005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381cac1-3173-4403-ac2f-adba24315028 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:25.633587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381cac1-3173-4403-ac2f-adba24315028 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:25.634983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381cac1-3173-4403-ac2f-adba24315028 service glance] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:25.646467 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1293/2554] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:25.722744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58611719-a7cd-4889-b409-def3199bbc22 None None] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:25.723033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58611719-a7cd-4889-b409-def3199bbc22 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:25.723313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58611719-a7cd-4889-b409-def3199bbc22 None None] PATH_INFO: `/` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:25.724666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1262/2555] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:59:25.729498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d635f-7aae-4ea2-99ee-2f3b544c280c None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:25.730379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d635f-7aae-4ea2-99ee-2f3b544c280c None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:25.730690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d635f-7aae-4ea2-99ee-2f3b544c280c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:25.774241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d12d635f-7aae-4ea2-99ee-2f3b544c280c None None] MFA Rules not processed for user `341277e7e3144e5ea93fa657916821e1`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:25.775460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d12d635f-7aae-4ea2-99ee-2f3b544c280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:25.782689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1294/2556] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:59:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:25.796941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58611719-a7cd-4889-b409-def3199bbc22 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:25.798673 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-58611719-a7cd-4889-b409-def3199bbc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:25.814409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58611719-a7cd-4889-b409-def3199bbc22 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:25.818087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20958362-b41c-4212-90e0-3e09627480fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:25.818815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958362-b41c-4212-90e0-3e09627480fd service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:25.819121 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958362-b41c-4212-90e0-3e09627480fd service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:25.819582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20958362-b41c-4212-90e0-3e09627480fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:25.828026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958362-b41c-4212-90e0-3e09627480fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:25.829787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20958362-b41c-4212-90e0-3e09627480fd service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:25.848513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1263/2557] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:25.938297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fa063-d814-4d7e-972d-bb5a961bea07 None None] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:25.938886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fa063-d814-4d7e-972d-bb5a961bea07 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:25.939164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fa063-d814-4d7e-972d-bb5a961bea07 None None] PATH_INFO: `/` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:25.940494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1295/2558] 10.222.0.15 () {54 vars in 921 bytes} [Fri Sep 9 01:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 01:59:25.946841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d8b0a-da6f-4562-b9d3-9c5bcfb8f9fd None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:25.947243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d8b0a-da6f-4562-b9d3-9c5bcfb8f9fd None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:25.947589 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d8b0a-da6f-4562-b9d3-9c5bcfb8f9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:25.968109 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d88d8b0a-da6f-4562-b9d3-9c5bcfb8f9fd None None] MFA Rules not processed for user `5a63d93adb4e4044ae79fb5aa7d8f5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:25.969410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d88d8b0a-da6f-4562-b9d3-9c5bcfb8f9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:25.979070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1264/2559] 10.222.0.15 () {58 vars in 1018 bytes} [Fri Sep 9 01:59:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:25.990490 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fa063-d814-4d7e-972d-bb5a961bea07 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:26.002477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fa063-d814-4d7e-972d-bb5a961bea07 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:26.006172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233ec2a-6672-4016-9213-76080082339c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:26.006765 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233ec2a-6672-4016-9213-76080082339c service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:26.007040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233ec2a-6672-4016-9213-76080082339c service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:26.007307 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233ec2a-6672-4016-9213-76080082339c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:26.018064 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233ec2a-6672-4016-9213-76080082339c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:26.019685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5233ec2a-6672-4016-9213-76080082339c service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:26.031394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1296/2560] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:26.106930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d8b0a-da6f-4562-b9d3-9c5bcfb8f9fd None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:26.108783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d88d8b0a-da6f-4562-b9d3-9c5bcfb8f9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:26.184140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d8b0a-da6f-4562-b9d3-9c5bcfb8f9fd None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:26.189387 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:26.190025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:26.190305 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:26.190577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:26.192299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:26.203536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:26.205173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:26.214958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1265/2561] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:28.805628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec144a54-1496-4597-b3f0-08e0278b8034 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:28.806084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec144a54-1496-4597-b3f0-08e0278b8034 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:28.806084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec144a54-1496-4597-b3f0-08e0278b8034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:28.824615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ec144a54-1496-4597-b3f0-08e0278b8034 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:28.825816 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ec144a54-1496-4597-b3f0-08e0278b8034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:28.856168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:28.880763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1297/2562] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 01:59:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:28.892803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:28.894243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:28.964541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2537b6-f6f6-49c4-8f91-b776d4840ea6 service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:28.969509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93f0a5-27f0-41a0-9763-21f9aa6e7c2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:28.970110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f0a5-27f0-41a0-9763-21f9aa6e7c2e service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:28.970384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f0a5-27f0-41a0-9763-21f9aa6e7c2e service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:28.970652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f0a5-27f0-41a0-9763-21f9aa6e7c2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:28.979241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93f0a5-27f0-41a0-9763-21f9aa6e7c2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:28.981463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93f0a5-27f0-41a0-9763-21f9aa6e7c2e service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:28.985081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:28.991289 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1266/2563] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 01:59:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 01:59:38.874258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=9 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:38.988515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:39.221546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec144a54-1496-4597-b3f0-08e0278b8034 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:39.288695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec144a54-1496-4597-b3f0-08e0278b8034 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:39.295349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:39.296078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:39.296421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:39.296770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] PATH_INFO: `/v3/users/91cf485ce0db45ba9597d28aa8768350` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:39.298017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91cf485ce0db45ba9597d28aa8768350)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:39.300491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:39.349869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91cf485ce0db45ba9597d28aa8768350'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:39.350624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91cf485ce0db45ba9597d28aa8768350'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:39.363697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91cf485ce0db45ba9597d28aa8768350'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:39.371730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91cf485ce0db45ba9597d28aa8768350'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:39.372351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91cf485ce0db45ba9597d28aa8768350'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:39.388210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1298/2564] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:59:39 2022] DELETE /identity/v3/users/91cf485ce0db45ba9597d28aa8768350 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:39.392611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da7645-3b6e-4089-a579-31bdafb963db None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:39.392898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da7645-3b6e-4089-a579-31bdafb963db None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:39.393213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da7645-3b6e-4089-a579-31bdafb963db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:39.412854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-38da7645-3b6e-4089-a579-31bdafb963db None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:39.414102 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-38da7645-3b6e-4089-a579-31bdafb963db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:39.477197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1267/2565] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:39.488235 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:39.489745 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:39.553159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611d68f-b003-4082-910f-b7eb1c8fd776 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:39.556784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dad60e-3d8d-4b8a-b8a4-af2d94a30bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:39.557940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dad60e-3d8d-4b8a-b8a4-af2d94a30bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:39.558212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dad60e-3d8d-4b8a-b8a4-af2d94a30bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:39.558483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dad60e-3d8d-4b8a-b8a4-af2d94a30bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:39.568403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dad60e-3d8d-4b8a-b8a4-af2d94a30bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:39.569859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dad60e-3d8d-4b8a-b8a4-af2d94a30bf1 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:39.579795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1299/2566] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:39.882365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da7645-3b6e-4089-a579-31bdafb963db None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:39.885954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-38da7645-3b6e-4089-a579-31bdafb963db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:39.911687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da7645-3b6e-4089-a579-31bdafb963db None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:39.928768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:39.930586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:39.930972 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:39.931340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] PATH_INFO: `/v3/projects/ab6a05d721c24383a92e3d239e782c2a` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:39.932707 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6a05d721c24383a92e3d239e782c2a)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:39.935171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:39.999043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6a05d721c24383a92e3d239e782c2a', 'request_id': 'req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.018039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6a05d721c24383a92e3d239e782c2a', 'request_id': 'req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.025528 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6a05d721c24383a92e3d239e782c2a', 'request_id': 'req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.046467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] The token cache is being invalidate because project ab6a05d721c24383a92e3d239e782c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:59:40.046803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.048084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.049012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1268/2567] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:59:39 2022] DELETE /identity/v3/projects/ab6a05d721c24383a92e3d239e782c2a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:40.052772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dad60e-3d8d-4b8a-b8a4-af2d94a30bf1 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:40.063711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dad60e-3d8d-4b8a-b8a4-af2d94a30bf1 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:40.083824 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:40.125664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:40.126306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:40.126585 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:40.126856 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] PATH_INFO: `/v3/users/b9eddbef315245d29cc399798a87616e` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:40.127971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9eddbef315245d29cc399798a87616e)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:40.129971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:40.183194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9eddbef315245d29cc399798a87616e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.183801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9eddbef315245d29cc399798a87616e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.196212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9eddbef315245d29cc399798a87616e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.209798 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9eddbef315245d29cc399798a87616e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.210584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-624e0ba4-f29a-4806-80b5-eb68c0818de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9eddbef315245d29cc399798a87616e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.224983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1300/2568] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:59:40 2022] DELETE /identity/v3/users/b9eddbef315245d29cc399798a87616e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:59:40.522231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:40.523843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:40.537481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a33f61-9f75-4009-9c4c-bf42fbaf6e68 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:40.594038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:40.594712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:40.595003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:40.595300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] PATH_INFO: `/v3/projects/820422e35dab4506adc06c761c7339c0` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:40.596604 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=820422e35dab4506adc06c761c7339c0)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:40.598662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:40.630768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:40.648386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820422e35dab4506adc06c761c7339c0', 'request_id': 'req-0102acc6-9980-4548-9d74-599a01559eae'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.654501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820422e35dab4506adc06c761c7339c0', 'request_id': 'req-0102acc6-9980-4548-9d74-599a01559eae'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.660753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820422e35dab4506adc06c761c7339c0', 'request_id': 'req-0102acc6-9980-4548-9d74-599a01559eae'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.674066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] The token cache is being invalidate because project 820422e35dab4506adc06c761c7339c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:59:40.674352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.674975 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-0102acc6-9980-4548-9d74-599a01559eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:40.679631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1269/2569] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:59:40 2022] DELETE /identity/v3/projects/820422e35dab4506adc06c761c7339c0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:40.681371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2ae4a-db72-45f8-a37d-79a3c942a948 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:40.681726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2ae4a-db72-45f8-a37d-79a3c942a948 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:40.682559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2ae4a-db72-45f8-a37d-79a3c942a948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:40.702712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c8f2ae4a-db72-45f8-a37d-79a3c942a948 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:40.703845 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c8f2ae4a-db72-45f8-a37d-79a3c942a948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:40.766280 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1301/2570] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:59:40.774028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf44bb-67c6-4bd1-9ef0-1e9fe887539d None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:40.774310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf44bb-67c6-4bd1-9ef0-1e9fe887539d None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:40.774608 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf44bb-67c6-4bd1-9ef0-1e9fe887539d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:40.794456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6acf44bb-67c6-4bd1-9ef0-1e9fe887539d None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:40.795652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6acf44bb-67c6-4bd1-9ef0-1e9fe887539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:40.804096 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1270/2571] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:40.809039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2ae4a-db72-45f8-a37d-79a3c942a948 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:40.819133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2ae4a-db72-45f8-a37d-79a3c942a948 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:40.824380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:40.824988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:40.825299 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:40.825574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:40.826273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:59:40.826519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:40.828187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:40.840602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1302/2572] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:59:40.845020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf44bb-67c6-4bd1-9ef0-1e9fe887539d None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:40.854202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf44bb-67c6-4bd1-9ef0-1e9fe887539d None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:40.858838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72189b-fd34-49ff-82b9-7fc87792ed0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:40.859427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72189b-fd34-49ff-82b9-7fc87792ed0b admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:40.859703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72189b-fd34-49ff-82b9-7fc87792ed0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:40.859974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72189b-fd34-49ff-82b9-7fc87792ed0b admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:40.860592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72189b-fd34-49ff-82b9-7fc87792ed0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:40.862505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72189b-fd34-49ff-82b9-7fc87792ed0b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:40.892536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1271/2573] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:40 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:40.896738 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:40.912838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57ae20-f457-4c61-8e2e-52b5d7225a60 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:40.940075 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:40.974027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81011671-fb48-466e-8372-74f6a698b0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:40.974552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81011671-fb48-466e-8372-74f6a698b0cb admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:40.974825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81011671-fb48-466e-8372-74f6a698b0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:40.975093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81011671-fb48-466e-8372-74f6a698b0cb admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:40.975720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81011671-fb48-466e-8372-74f6a698b0cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:40.977750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81011671-fb48-466e-8372-74f6a698b0cb admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.003279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1303/2574] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:40 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:41.004536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72189b-fd34-49ff-82b9-7fc87792ed0b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.015746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72189b-fd34-49ff-82b9-7fc87792ed0b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.020501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb91a2-338f-40ce-ad71-f56ecf4052bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.021312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb91a2-338f-40ce-ad71-f56ecf4052bc admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.021655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb91a2-338f-40ce-ad71-f56ecf4052bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.021988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb91a2-338f-40ce-ad71-f56ecf4052bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.022690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb91a2-338f-40ce-ad71-f56ecf4052bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.024307 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb91a2-338f-40ce-ad71-f56ecf4052bc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.038112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1272/2575] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:41.039237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81011671-fb48-466e-8372-74f6a698b0cb admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.049099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81011671-fb48-466e-8372-74f6a698b0cb admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.053697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de0ba1-2898-4639-ad00-86c6a250fbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.054318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de0ba1-2898-4639-ad00-86c6a250fbfd admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.054553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de0ba1-2898-4639-ad00-86c6a250fbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.054821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de0ba1-2898-4639-ad00-86c6a250fbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.055882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de0ba1-2898-4639-ad00-86c6a250fbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.057638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de0ba1-2898-4639-ad00-86c6a250fbfd admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.069669 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1304/2576] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:41.074543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb91a2-338f-40ce-ad71-f56ecf4052bc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.085368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb91a2-338f-40ce-ad71-f56ecf4052bc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.090739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f674f-6a95-4eaa-b495-93c116bc304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.091401 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f674f-6a95-4eaa-b495-93c116bc304b admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.091678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f674f-6a95-4eaa-b495-93c116bc304b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.091991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f674f-6a95-4eaa-b495-93c116bc304b admin admin] PATH_INFO: `/v3/projects/57ed12abd9f94b28a0cf3c8de3e10172/users/c91b2fa5ad2c43eab064231033c9a710/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.111213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f674f-6a95-4eaa-b495-93c116bc304b admin admin] RBAC: Authorizing `identity:create_grant(project_id=57ed12abd9f94b28a0cf3c8de3e10172, user_id=c91b2fa5ad2c43eab064231033c9a710, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.114318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f674f-6a95-4eaa-b495-93c116bc304b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.135404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1273/2577] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:41 2022] PUT /identity/v3/projects/57ed12abd9f94b28a0cf3c8de3e10172/users/c91b2fa5ad2c43eab064231033c9a710/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:59:41.145855 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144503d-30e7-4c83-9174-562c7406c275 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.146224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144503d-30e7-4c83-9174-562c7406c275 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.146563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144503d-30e7-4c83-9174-562c7406c275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.177651 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-b144503d-30e7-4c83-9174-562c7406c275 None None] MFA Rules not processed for user `c91b2fa5ad2c43eab064231033c9a710`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:41.178957 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b144503d-30e7-4c83-9174-562c7406c275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:41.251049 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1305/2578] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:41.255485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f674f-6a95-4eaa-b495-93c116bc304b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.318054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f674f-6a95-4eaa-b495-93c116bc304b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.322791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45980284-d468-4e55-8f14-effd55d7703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.323402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45980284-d468-4e55-8f14-effd55d7703a admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.323683 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45980284-d468-4e55-8f14-effd55d7703a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.323957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45980284-d468-4e55-8f14-effd55d7703a admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.324586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45980284-d468-4e55-8f14-effd55d7703a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.326544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45980284-d468-4e55-8f14-effd55d7703a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.370838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1274/2579] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:41 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:41.374911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144503d-30e7-4c83-9174-562c7406c275 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.386883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144503d-30e7-4c83-9174-562c7406c275 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.444852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c40fd-4dfc-461f-97d2-61f20e352478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.445498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c40fd-4dfc-461f-97d2-61f20e352478 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.445781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c40fd-4dfc-461f-97d2-61f20e352478 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.446047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c40fd-4dfc-461f-97d2-61f20e352478 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.446681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c40fd-4dfc-461f-97d2-61f20e352478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.448570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c40fd-4dfc-461f-97d2-61f20e352478 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.471089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1306/2580] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:41 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:41.482749 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45980284-d468-4e55-8f14-effd55d7703a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.495834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45980284-d468-4e55-8f14-effd55d7703a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.500663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88263f95-107c-4410-879b-1ee93c192f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.501365 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88263f95-107c-4410-879b-1ee93c192f12 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.501629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88263f95-107c-4410-879b-1ee93c192f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.502091 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88263f95-107c-4410-879b-1ee93c192f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.503157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88263f95-107c-4410-879b-1ee93c192f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.505959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88263f95-107c-4410-879b-1ee93c192f12 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.522645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1275/2581] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:41.527370 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c40fd-4dfc-461f-97d2-61f20e352478 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.537745 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c40fd-4dfc-461f-97d2-61f20e352478 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.543116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bd10e-f9c5-4aed-b2f3-c3946d0be289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.543774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd10e-f9c5-4aed-b2f3-c3946d0be289 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.544054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd10e-f9c5-4aed-b2f3-c3946d0be289 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.544329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd10e-f9c5-4aed-b2f3-c3946d0be289 admin admin] PATH_INFO: `/v3/projects/9b600c4be1264bd1bdc4015b8a3d2bc0/users/089ff81962b749ecaebc24cc6dcb8f58/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.558878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bd10e-f9c5-4aed-b2f3-c3946d0be289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b600c4be1264bd1bdc4015b8a3d2bc0, user_id=089ff81962b749ecaebc24cc6dcb8f58, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.561557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bd10e-f9c5-4aed-b2f3-c3946d0be289 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.574440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1307/2582] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:41 2022] PUT /identity/v3/projects/9b600c4be1264bd1bdc4015b8a3d2bc0/users/089ff81962b749ecaebc24cc6dcb8f58/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:41.578451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88263f95-107c-4410-879b-1ee93c192f12 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.588971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88263f95-107c-4410-879b-1ee93c192f12 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.609765 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:41.648354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386473b-a035-4fc7-8fc0-7d58a88247bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.648951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386473b-a035-4fc7-8fc0-7d58a88247bd admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.649258 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386473b-a035-4fc7-8fc0-7d58a88247bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.649525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386473b-a035-4fc7-8fc0-7d58a88247bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.650144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386473b-a035-4fc7-8fc0-7d58a88247bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.652340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386473b-a035-4fc7-8fc0-7d58a88247bd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.661971 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1276/2583] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:41.666806 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bd10e-f9c5-4aed-b2f3-c3946d0be289 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.672154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:41.678510 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bd10e-f9c5-4aed-b2f3-c3946d0be289 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.683434 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.684063 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.684337 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.684606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] PATH_INFO: `/v3/projects/9b600c4be1264bd1bdc4015b8a3d2bc0/users/089ff81962b749ecaebc24cc6dcb8f58/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.686568 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b600c4be1264bd1bdc4015b8a3d2bc0, user_id=089ff81962b749ecaebc24cc6dcb8f58, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.689312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.699310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1308/2584] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:41 2022] PUT /identity/v3/projects/9b600c4be1264bd1bdc4015b8a3d2bc0/users/089ff81962b749ecaebc24cc6dcb8f58/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:41.710210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175be5b-eb88-4b74-af9f-e842edf68bef None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.710495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175be5b-eb88-4b74-af9f-e842edf68bef None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.710767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175be5b-eb88-4b74-af9f-e842edf68bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.739535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0175be5b-eb88-4b74-af9f-e842edf68bef None None] MFA Rules not processed for user `089ff81962b749ecaebc24cc6dcb8f58`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:41.741004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0175be5b-eb88-4b74-af9f-e842edf68bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:41.807656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1277/2585] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:41.818769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:41.820448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:41.883371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c776fe-785c-4776-b170-818cd79f0a64 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:41.887598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a6f7c-354e-427f-a0d5-4fbac7cccaa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:41.888193 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a6f7c-354e-427f-a0d5-4fbac7cccaa7 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:41.888468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a6f7c-354e-427f-a0d5-4fbac7cccaa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:41.888742 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a6f7c-354e-427f-a0d5-4fbac7cccaa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:41.899401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a6f7c-354e-427f-a0d5-4fbac7cccaa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:41.901544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a6f7c-354e-427f-a0d5-4fbac7cccaa7 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:41.912706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1309/2586] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:41.998502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175be5b-eb88-4b74-af9f-e842edf68bef None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:42.008102 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175be5b-eb88-4b74-af9f-e842edf68bef None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:42.012292 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38ea09-0fd2-4aa4-9f16-85b4909a2540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:42.012880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ea09-0fd2-4aa4-9f16-85b4909a2540 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:42.013186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ea09-0fd2-4aa4-9f16-85b4909a2540 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:42.013457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38ea09-0fd2-4aa4-9f16-85b4909a2540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:42.068343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ea09-0fd2-4aa4-9f16-85b4909a2540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:42.069803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38ea09-0fd2-4aa4-9f16-85b4909a2540 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:42.079394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1278/2587] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:51.908199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:52.075360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=80 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:56.651977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a6f7c-354e-427f-a0d5-4fbac7cccaa7 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:56.713066 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a6f7c-354e-427f-a0d5-4fbac7cccaa7 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:56.717676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:56.718356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:56.718577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:56.718847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] PATH_INFO: `/v3/users/c91b2fa5ad2c43eab064231033c9a710` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:56.719961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91b2fa5ad2c43eab064231033c9a710)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:56.721913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:56.768323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91b2fa5ad2c43eab064231033c9a710'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:56.769017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91b2fa5ad2c43eab064231033c9a710'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:56.781414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91b2fa5ad2c43eab064231033c9a710'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:56.788553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91b2fa5ad2c43eab064231033c9a710'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:56.789204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91b2fa5ad2c43eab064231033c9a710'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:56.801602 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1310/2588] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:59:56 2022] DELETE /identity/v3/users/c91b2fa5ad2c43eab064231033c9a710 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:56.805981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53248c-23c3-4e50-9c52-00b9a27b8ef7 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:56.806487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53248c-23c3-4e50-9c52-00b9a27b8ef7 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:56.806602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53248c-23c3-4e50-9c52-00b9a27b8ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:56.825635 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-7a53248c-23c3-4e50-9c52-00b9a27b8ef7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:56.826834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7a53248c-23c3-4e50-9c52-00b9a27b8ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:56.888321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1279/2589] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:56.898048 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:56.899447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:56.956908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4b71a-a240-48f2-b249-b827569cb11c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:56.960531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d5857-f4dc-418e-9315-c1315b8ecb75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:56.961162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5857-f4dc-418e-9315-c1315b8ecb75 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:56.961433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5857-f4dc-418e-9315-c1315b8ecb75 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:56.961701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5857-f4dc-418e-9315-c1315b8ecb75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:56.971065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d5857-f4dc-418e-9315-c1315b8ecb75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:56.972415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d5857-f4dc-418e-9315-c1315b8ecb75 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:56.981547 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1311/2590] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:57.242536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53248c-23c3-4e50-9c52-00b9a27b8ef7 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:57.244156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7a53248c-23c3-4e50-9c52-00b9a27b8ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:57.257965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53248c-23c3-4e50-9c52-00b9a27b8ef7 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:57.262607 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:57.263717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:57.264163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:57.264443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] PATH_INFO: `/v3/projects/57ed12abd9f94b28a0cf3c8de3e10172` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:57.265763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57ed12abd9f94b28a0cf3c8de3e10172)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:57.267554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:57.313739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57ed12abd9f94b28a0cf3c8de3e10172', 'request_id': 'req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.319447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57ed12abd9f94b28a0cf3c8de3e10172', 'request_id': 'req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.325626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57ed12abd9f94b28a0cf3c8de3e10172', 'request_id': 'req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.338108 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] The token cache is being invalidate because project 57ed12abd9f94b28a0cf3c8de3e10172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:59:57.338397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.339017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.340493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1280/2591] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:59:57 2022] DELETE /identity/v3/projects/57ed12abd9f94b28a0cf3c8de3e10172 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:57.344445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d5857-f4dc-418e-9315-c1315b8ecb75 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:57.354706 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d5857-f4dc-418e-9315-c1315b8ecb75 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:57.372590 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:57.413366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:57.413994 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:57.414274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:57.414548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] PATH_INFO: `/v3/users/089ff81962b749ecaebc24cc6dcb8f58` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:57.415717 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] RBAC: Authorizing `identity:delete_user(user_id=089ff81962b749ecaebc24cc6dcb8f58)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:57.417925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:57.464090 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '089ff81962b749ecaebc24cc6dcb8f58'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.464744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '089ff81962b749ecaebc24cc6dcb8f58'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.476648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '089ff81962b749ecaebc24cc6dcb8f58'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.483282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '089ff81962b749ecaebc24cc6dcb8f58'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.483987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-794d755f-2046-462e-98c6-fb41ec52d26f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '089ff81962b749ecaebc24cc6dcb8f58'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.496894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1312/2592] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 01:59:57 2022] DELETE /identity/v3/users/089ff81962b749ecaebc24cc6dcb8f58 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:57.766953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:57.768804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:57.785783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bf88b-e20a-4338-a2e7-7d77bd1822d3 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:57.849795 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:57.850419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:57.850687 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:57.850959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] PATH_INFO: `/v3/projects/9b600c4be1264bd1bdc4015b8a3d2bc0` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:57.852065 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b600c4be1264bd1bdc4015b8a3d2bc0)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:57.854512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:57.884962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:57.903353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b600c4be1264bd1bdc4015b8a3d2bc0', 'request_id': 'req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.908829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b600c4be1264bd1bdc4015b8a3d2bc0', 'request_id': 'req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.915238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b600c4be1264bd1bdc4015b8a3d2bc0', 'request_id': 'req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.927631 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] The token cache is being invalidate because project 9b600c4be1264bd1bdc4015b8a3d2bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 01:59:57.927915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.928549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-60b36ef2-eb83-4cb6-b147-63ddcddc5a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 01:59:57.930142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1281/2593] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 01:59:57 2022] DELETE /identity/v3/projects/9b600c4be1264bd1bdc4015b8a3d2bc0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:57.935518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0145-b725-4cd6-b895-fbc45d022372 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:57.935963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0145-b725-4cd6-b895-fbc45d022372 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:57.936058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e0145-b725-4cd6-b895-fbc45d022372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:57.955610 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-017e0145-b725-4cd6-b895-fbc45d022372 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:57.956778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-017e0145-b725-4cd6-b895-fbc45d022372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:58.013690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1313/2594] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 01:59:58.021434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d33b6-c2ce-4cce-8bc2-46e1dff724d4 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.021717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d33b6-c2ce-4cce-8bc2-46e1dff724d4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.021989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d33b6-c2ce-4cce-8bc2-46e1dff724d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.041085 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e77d33b6-c2ce-4cce-8bc2-46e1dff724d4 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:58.042282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e77d33b6-c2ce-4cce-8bc2-46e1dff724d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:58.050106 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1282/2595] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:58.054135 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e0145-b725-4cd6-b895-fbc45d022372 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.064435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e0145-b725-4cd6-b895-fbc45d022372 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.069153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.069753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.070030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.070302 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.070940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 01:59:58.071242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.073461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.085213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1314/2596] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 01:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 01:59:58.089494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d33b6-c2ce-4cce-8bc2-46e1dff724d4 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.098606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d33b6-c2ce-4cce-8bc2-46e1dff724d4 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.103143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f051b44-e2e6-40d9-aa38-c6b7a10b81e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.103724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f051b44-e2e6-40d9-aa38-c6b7a10b81e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.103999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f051b44-e2e6-40d9-aa38-c6b7a10b81e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.104265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f051b44-e2e6-40d9-aa38-c6b7a10b81e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.104875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f051b44-e2e6-40d9-aa38-c6b7a10b81e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.106788 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f051b44-e2e6-40d9-aa38-c6b7a10b81e8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.133503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1283/2597] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:58 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:58.138213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.147847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f062c-466f-4adb-851b-4b4bf2322e87 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.172952 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:58.204863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8abc-902f-422f-b188-23da63b622e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.205496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8abc-902f-422f-b188-23da63b622e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.205771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8abc-902f-422f-b188-23da63b622e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.206040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8abc-902f-422f-b188-23da63b622e7 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.206656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad8abc-902f-422f-b188-23da63b622e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.208548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad8abc-902f-422f-b188-23da63b622e7 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.235978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1315/2598] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:58 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:58.237204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f051b44-e2e6-40d9-aa38-c6b7a10b81e8 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.248084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f051b44-e2e6-40d9-aa38-c6b7a10b81e8 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.252830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3974417-94c7-494c-8a20-8790249e5d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.253452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974417-94c7-494c-8a20-8790249e5d3c admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.253727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974417-94c7-494c-8a20-8790249e5d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.253994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974417-94c7-494c-8a20-8790249e5d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.254620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3974417-94c7-494c-8a20-8790249e5d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.256194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3974417-94c7-494c-8a20-8790249e5d3c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.266981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1284/2599] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:58.270895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8abc-902f-422f-b188-23da63b622e7 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.281085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad8abc-902f-422f-b188-23da63b622e7 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.285675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0ace3-fe59-4c3a-88ef-a02394523776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.286258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ace3-fe59-4c3a-88ef-a02394523776 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.286533 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ace3-fe59-4c3a-88ef-a02394523776 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.286802 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ace3-fe59-4c3a-88ef-a02394523776 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.287427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0ace3-fe59-4c3a-88ef-a02394523776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.288985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0ace3-fe59-4c3a-88ef-a02394523776 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.299777 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1316/2600] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:58.303993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3974417-94c7-494c-8a20-8790249e5d3c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.314452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3974417-94c7-494c-8a20-8790249e5d3c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.319035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3816f52-f93d-4289-b785-374ead7c8a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.319673 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3816f52-f93d-4289-b785-374ead7c8a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.319951 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3816f52-f93d-4289-b785-374ead7c8a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.320218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3816f52-f93d-4289-b785-374ead7c8a4e admin admin] PATH_INFO: `/v3/projects/bedd718575c54661a2225ce60fe9925c/users/af011ce002144d34a267b3c2e89e1fae/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.335002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3816f52-f93d-4289-b785-374ead7c8a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bedd718575c54661a2225ce60fe9925c, user_id=af011ce002144d34a267b3c2e89e1fae, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.337656 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3816f52-f93d-4289-b785-374ead7c8a4e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.350751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1285/2601] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:58 2022] PUT /identity/v3/projects/bedd718575c54661a2225ce60fe9925c/users/af011ce002144d34a267b3c2e89e1fae/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:58.362289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36a3a-a2cf-452c-bf96-552a2b1e875d None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.362598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36a3a-a2cf-452c-bf96-552a2b1e875d None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.362868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36a3a-a2cf-452c-bf96-552a2b1e875d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.392405 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-06d36a3a-a2cf-452c-bf96-552a2b1e875d None None] MFA Rules not processed for user `af011ce002144d34a267b3c2e89e1fae`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:58.393628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-06d36a3a-a2cf-452c-bf96-552a2b1e875d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:58.455868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1317/2602] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:58.460382 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3816f52-f93d-4289-b785-374ead7c8a4e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.521809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3816f52-f93d-4289-b785-374ead7c8a4e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.526827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7705c-d26b-4e55-9e7b-9c6467d337a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.527418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7705c-d26b-4e55-9e7b-9c6467d337a6 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.527690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7705c-d26b-4e55-9e7b-9c6467d337a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.527956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7705c-d26b-4e55-9e7b-9c6467d337a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.528564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7705c-d26b-4e55-9e7b-9c6467d337a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.530474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7705c-d26b-4e55-9e7b-9c6467d337a6 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.555540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1286/2603] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 01:59:58 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:58.559623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36a3a-a2cf-452c-bf96-552a2b1e875d None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.570394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36a3a-a2cf-452c-bf96-552a2b1e875d None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.625548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb1fbc-4b6a-4ef8-a93c-e37fc506b950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.626144 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb1fbc-4b6a-4ef8-a93c-e37fc506b950 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.626423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb1fbc-4b6a-4ef8-a93c-e37fc506b950 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.626696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb1fbc-4b6a-4ef8-a93c-e37fc506b950 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.627308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb1fbc-4b6a-4ef8-a93c-e37fc506b950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.629239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb1fbc-4b6a-4ef8-a93c-e37fc506b950 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.650386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1318/2604] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 01:59:58 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 01:59:58.654301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7705c-d26b-4e55-9e7b-9c6467d337a6 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.664352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7705c-d26b-4e55-9e7b-9c6467d337a6 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.669688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cee89-4fef-44b6-a2d6-16b3da627161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.670282 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cee89-4fef-44b6-a2d6-16b3da627161 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.670564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cee89-4fef-44b6-a2d6-16b3da627161 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.670834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cee89-4fef-44b6-a2d6-16b3da627161 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.671475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cee89-4fef-44b6-a2d6-16b3da627161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.673120 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cee89-4fef-44b6-a2d6-16b3da627161 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.684481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1287/2605] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:58.688238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb1fbc-4b6a-4ef8-a93c-e37fc506b950 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.697866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb1fbc-4b6a-4ef8-a93c-e37fc506b950 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.702349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c071f4-d32b-4153-83ab-6641a1b5bce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.702990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c071f4-d32b-4153-83ab-6641a1b5bce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.703265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c071f4-d32b-4153-83ab-6641a1b5bce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.703535 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c071f4-d32b-4153-83ab-6641a1b5bce9 admin admin] PATH_INFO: `/v3/projects/03cddc70caed477bbd4a8b012ae730c9/users/dd6c5eb6d710468cbf2dd2476117000b/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.718300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c071f4-d32b-4153-83ab-6641a1b5bce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cddc70caed477bbd4a8b012ae730c9, user_id=dd6c5eb6d710468cbf2dd2476117000b, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.720943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c071f4-d32b-4153-83ab-6641a1b5bce9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.733292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1319/2606] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:58 2022] PUT /identity/v3/projects/03cddc70caed477bbd4a8b012ae730c9/users/dd6c5eb6d710468cbf2dd2476117000b/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 01:59:58.737033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cee89-4fef-44b6-a2d6-16b3da627161 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.747511 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cee89-4fef-44b6-a2d6-16b3da627161 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.766969 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:58.804032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa564ea5-aed7-47fc-a040-daed897eb03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.804555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa564ea5-aed7-47fc-a040-daed897eb03a admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.804830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa564ea5-aed7-47fc-a040-daed897eb03a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.805125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa564ea5-aed7-47fc-a040-daed897eb03a admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.805747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa564ea5-aed7-47fc-a040-daed897eb03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.807804 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa564ea5-aed7-47fc-a040-daed897eb03a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.817341 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1288/2607] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 01:59:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 01:59:58.821542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c071f4-d32b-4153-83ab-6641a1b5bce9 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.827445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 01:59:58.833358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c071f4-d32b-4153-83ab-6641a1b5bce9 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:58.839102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:58.839761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.840040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.840313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] PATH_INFO: `/v3/projects/03cddc70caed477bbd4a8b012ae730c9/users/dd6c5eb6d710468cbf2dd2476117000b/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.842243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03cddc70caed477bbd4a8b012ae730c9, user_id=dd6c5eb6d710468cbf2dd2476117000b, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:58.844898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:58.856915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1320/2608] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 01:59:58 2022] PUT /identity/v3/projects/03cddc70caed477bbd4a8b012ae730c9/users/dd6c5eb6d710468cbf2dd2476117000b/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 01:59:58.867599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27d237-2bd8-440b-8859-c6a10adfd690 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:58.868107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27d237-2bd8-440b-8859-c6a10adfd690 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:58.868290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27d237-2bd8-440b-8859-c6a10adfd690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:58.898820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ef27d237-2bd8-440b-8859-c6a10adfd690 None None] MFA Rules not processed for user `dd6c5eb6d710468cbf2dd2476117000b`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 01:59:58.900070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ef27d237-2bd8-440b-8859-c6a10adfd690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:58.970985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1289/2609] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 01:59:58 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 01:59:58.983089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:58.984958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:59.043879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60670b-c883-4fe8-835a-769d05d0c409 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:59.047515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96197a1-51dc-4ac3-b4d4-1c175733289c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:59.048108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96197a1-51dc-4ac3-b4d4-1c175733289c service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:59.048386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96197a1-51dc-4ac3-b4d4-1c175733289c service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:59.048655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96197a1-51dc-4ac3-b4d4-1c175733289c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:59.057892 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96197a1-51dc-4ac3-b4d4-1c175733289c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:59.059238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96197a1-51dc-4ac3-b4d4-1c175733289c service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:59.068634 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1321/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:58 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 01:59:59.115808 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27d237-2bd8-440b-8859-c6a10adfd690 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 01:59:59.117227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ef27d237-2bd8-440b-8859-c6a10adfd690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 01:59:59.128861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27d237-2bd8-440b-8859-c6a10adfd690 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 01:59:59.132386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807997d3-fea3-447e-946b-e25a0551aa5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 01:59:59.132977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807997d3-fea3-447e-946b-e25a0551aa5b service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 01:59:59.133302 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807997d3-fea3-447e-946b-e25a0551aa5b service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 01:59:59.133573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807997d3-fea3-447e-946b-e25a0551aa5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 01:59:59.189661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807997d3-fea3-447e-946b-e25a0551aa5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 01:59:59.191080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807997d3-fea3-447e-946b-e25a0551aa5b service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 01:59:59.200452 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1290/2611] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 01:59:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:02.472963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96197a1-51dc-4ac3-b4d4-1c175733289c service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:02.537194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96197a1-51dc-4ac3-b4d4-1c175733289c service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:02.542270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:02.542901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:02.543257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:02.543440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] PATH_INFO: `/v3/users/af011ce002144d34a267b3c2e89e1fae` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:02.544614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] RBAC: Authorizing `identity:delete_user(user_id=af011ce002144d34a267b3c2e89e1fae)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:02.546546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:02.592115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af011ce002144d34a267b3c2e89e1fae'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:02.592780 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af011ce002144d34a267b3c2e89e1fae'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:02.605226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af011ce002144d34a267b3c2e89e1fae'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:02.612511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af011ce002144d34a267b3c2e89e1fae'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:02.613169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af011ce002144d34a267b3c2e89e1fae'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:02.625472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1322/2612] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:00:02 2022] DELETE /identity/v3/users/af011ce002144d34a267b3c2e89e1fae => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:02.629940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ce94c-49ee-482a-b48e-c8fa80bc83a4 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:02.630396 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ce94c-49ee-482a-b48e-c8fa80bc83a4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:02.630512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ce94c-49ee-482a-b48e-c8fa80bc83a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:02.649136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e19ce94c-49ee-482a-b48e-c8fa80bc83a4 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:02.650880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e19ce94c-49ee-482a-b48e-c8fa80bc83a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:02.690385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:02.710252 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1291/2613] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:02.720254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:02.721707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:02.735695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:02.781218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380664-fad1-4a04-9685-a1949295065f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:02.784936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0863b3-0fa1-42dd-96f1-79b2f779ab47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:02.785588 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0863b3-0fa1-42dd-96f1-79b2f779ab47 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:02.785869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0863b3-0fa1-42dd-96f1-79b2f779ab47 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:02.786167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0863b3-0fa1-42dd-96f1-79b2f779ab47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:02.795284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0863b3-0fa1-42dd-96f1-79b2f779ab47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:02.796700 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0863b3-0fa1-42dd-96f1-79b2f779ab47 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:02.806534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1323/2614] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:03.141430 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ce94c-49ee-482a-b48e-c8fa80bc83a4 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:03.143007 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e19ce94c-49ee-482a-b48e-c8fa80bc83a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:03.155549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ce94c-49ee-482a-b48e-c8fa80bc83a4 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:03.160246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:03.160911 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:03.161228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:03.161503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] PATH_INFO: `/v3/projects/bedd718575c54661a2225ce60fe9925c` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:03.162732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bedd718575c54661a2225ce60fe9925c)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:03.164633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:03.213119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bedd718575c54661a2225ce60fe9925c', 'request_id': 'req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.218603 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bedd718575c54661a2225ce60fe9925c', 'request_id': 'req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.224648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bedd718575c54661a2225ce60fe9925c', 'request_id': 'req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.236745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] The token cache is being invalidate because project bedd718575c54661a2225ce60fe9925c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:00:03.237033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.237681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.239164 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1292/2615] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:00:03 2022] DELETE /identity/v3/projects/bedd718575c54661a2225ce60fe9925c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:03.244628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0863b3-0fa1-42dd-96f1-79b2f779ab47 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:03.258014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0863b3-0fa1-42dd-96f1-79b2f779ab47 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:03.313741 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:03.314465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:03.314752 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:03.315022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] PATH_INFO: `/v3/users/dd6c5eb6d710468cbf2dd2476117000b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:03.316152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6c5eb6d710468cbf2dd2476117000b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:03.318082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:03.365936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6c5eb6d710468cbf2dd2476117000b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.366598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6c5eb6d710468cbf2dd2476117000b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.380382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6c5eb6d710468cbf2dd2476117000b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.389030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6c5eb6d710468cbf2dd2476117000b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.389763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6c5eb6d710468cbf2dd2476117000b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.403735 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1324/2616] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:00:03 2022] DELETE /identity/v3/users/dd6c5eb6d710468cbf2dd2476117000b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:03.460743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:03.463008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:03.552936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51d9d5-76fe-48b0-bf9b-8adb2a025b69 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:03.620072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:03.636038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3c92a-363d-4480-8236-91c65c2a38bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:03.639938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3c92a-363d-4480-8236-91c65c2a38bc service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:03.640266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3c92a-363d-4480-8236-91c65c2a38bc service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:03.640579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3c92a-363d-4480-8236-91c65c2a38bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:03.642532 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-08f3c92a-363d-4480-8236-91c65c2a38bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:03.658100 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3c92a-363d-4480-8236-91c65c2a38bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:03.660009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3c92a-363d-4480-8236-91c65c2a38bc service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:03.673418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1293/2617] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:03.721951 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:03.724097 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:03.785945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:03.819784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222d4ad-96b8-4aba-9d6a-d035c27757a4 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:03.825317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:03.825977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:03.826256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:03.826559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] PATH_INFO: `/v3/projects/03cddc70caed477bbd4a8b012ae730c9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:03.827770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03cddc70caed477bbd4a8b012ae730c9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:03.829769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:03.889712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03cddc70caed477bbd4a8b012ae730c9', 'request_id': 'req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.899081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03cddc70caed477bbd4a8b012ae730c9', 'request_id': 'req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.906072 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03cddc70caed477bbd4a8b012ae730c9', 'request_id': 'req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.923954 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] The token cache is being invalidate because project 03cddc70caed477bbd4a8b012ae730c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:00:03.924161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.924886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a71f13d7-8bf9-43ed-b9ad-03094cc3b9e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:03.930456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1325/2618] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:00:03 2022] DELETE /identity/v3/projects/03cddc70caed477bbd4a8b012ae730c9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:03.932715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86c94-ded4-4f0e-9c74-fe59f7ec1c39 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:03.933090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86c94-ded4-4f0e-9c74-fe59f7ec1c39 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:03.933464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef86c94-ded4-4f0e-9c74-fe59f7ec1c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:03.954743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-9ef86c94-ded4-4f0e-9c74-fe59f7ec1c39 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:03.955942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9ef86c94-ded4-4f0e-9c74-fe59f7ec1c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:04.021274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1294/2619] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:04.029725 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010d49-b382-480a-9066-cb5c4a1147aa None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.030001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010d49-b382-480a-9066-cb5c4a1147aa None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.030270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db010d49-b382-480a-9066-cb5c4a1147aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.051566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-db010d49-b382-480a-9066-cb5c4a1147aa None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:04.052798 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-db010d49-b382-480a-9066-cb5c4a1147aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:04.061094 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1326/2620] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:04.065749 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86c94-ded4-4f0e-9c74-fe59f7ec1c39 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.075871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef86c94-ded4-4f0e-9c74-fe59f7ec1c39 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.080592 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.081245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.081520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.081791 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.082441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:00:04.082739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.084349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.101046 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1295/2621] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 02:00:04.103206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db010d49-b382-480a-9066-cb5c4a1147aa None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.112772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db010d49-b382-480a-9066-cb5c4a1147aa None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.117921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b65e6-96f6-4b63-ba68-5814d527252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.118636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b65e6-96f6-4b63-ba68-5814d527252c admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.118983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b65e6-96f6-4b63-ba68-5814d527252c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.119361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b65e6-96f6-4b63-ba68-5814d527252c admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.120058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b65e6-96f6-4b63-ba68-5814d527252c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.122071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b65e6-96f6-4b63-ba68-5814d527252c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.153106 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1327/2622] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:00:04 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:04.155626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.166265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045316bc-e27a-4042-987c-c951f9f2c759 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.221586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddbb2b-43eb-4cf0-bfa1-912939550f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.222110 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddbb2b-43eb-4cf0-bfa1-912939550f02 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.222384 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddbb2b-43eb-4cf0-bfa1-912939550f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.222649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddbb2b-43eb-4cf0-bfa1-912939550f02 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.223271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddbb2b-43eb-4cf0-bfa1-912939550f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.225245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddbb2b-43eb-4cf0-bfa1-912939550f02 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.248116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1296/2623] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:00:04 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:04.252289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b65e6-96f6-4b63-ba68-5814d527252c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.262876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b65e6-96f6-4b63-ba68-5814d527252c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.268447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc5477-f0c2-4f8a-8ef3-6b1bcc9a7511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.269078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc5477-f0c2-4f8a-8ef3-6b1bcc9a7511 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.269351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc5477-f0c2-4f8a-8ef3-6b1bcc9a7511 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.269629 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc5477-f0c2-4f8a-8ef3-6b1bcc9a7511 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.270782 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc5477-f0c2-4f8a-8ef3-6b1bcc9a7511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.273462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc5477-f0c2-4f8a-8ef3-6b1bcc9a7511 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.285904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1328/2624] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:04.290670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddbb2b-43eb-4cf0-bfa1-912939550f02 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.305634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddbb2b-43eb-4cf0-bfa1-912939550f02 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.310381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc5cc8-efcd-4905-986a-452e86a75e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.310986 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc5cc8-efcd-4905-986a-452e86a75e35 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.311264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc5cc8-efcd-4905-986a-452e86a75e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.311535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc5cc8-efcd-4905-986a-452e86a75e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.312179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc5cc8-efcd-4905-986a-452e86a75e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.313823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc5cc8-efcd-4905-986a-452e86a75e35 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.325513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1297/2625] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:04.330169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc5477-f0c2-4f8a-8ef3-6b1bcc9a7511 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.341805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc5477-f0c2-4f8a-8ef3-6b1bcc9a7511 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.346390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf34724-3824-4906-b7fc-664b2015e24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.347027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf34724-3824-4906-b7fc-664b2015e24d admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.347304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf34724-3824-4906-b7fc-664b2015e24d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.347572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf34724-3824-4906-b7fc-664b2015e24d admin admin] PATH_INFO: `/v3/projects/6e6e12cd026c408197d5593df3d9363d/users/aa1e2e4bc25148bc80e0aff5efcc9b55/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.363306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf34724-3824-4906-b7fc-664b2015e24d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6e12cd026c408197d5593df3d9363d, user_id=aa1e2e4bc25148bc80e0aff5efcc9b55, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.366217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf34724-3824-4906-b7fc-664b2015e24d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.380320 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1329/2626] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:00:04 2022] PUT /identity/v3/projects/6e6e12cd026c408197d5593df3d9363d/users/aa1e2e4bc25148bc80e0aff5efcc9b55/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:04.451066 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8761244-49f3-4b03-a134-545bcfa55dae None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.451481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8761244-49f3-4b03-a134-545bcfa55dae None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.451752 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8761244-49f3-4b03-a134-545bcfa55dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.479855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e8761244-49f3-4b03-a134-545bcfa55dae None None] MFA Rules not processed for user `aa1e2e4bc25148bc80e0aff5efcc9b55`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:04.481361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e8761244-49f3-4b03-a134-545bcfa55dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:04.526562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:04.547331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1298/2627] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:04.551761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf34724-3824-4906-b7fc-664b2015e24d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.615718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf34724-3824-4906-b7fc-664b2015e24d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.620487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641584f-1e7e-465b-b8e0-4de584d74cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.621109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641584f-1e7e-465b-b8e0-4de584d74cec admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.621392 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641584f-1e7e-465b-b8e0-4de584d74cec admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.621667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641584f-1e7e-465b-b8e0-4de584d74cec admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.622338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641584f-1e7e-465b-b8e0-4de584d74cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.624220 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641584f-1e7e-465b-b8e0-4de584d74cec admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.628606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:04.651817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1330/2628] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:00:04 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:04.656090 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8761244-49f3-4b03-a134-545bcfa55dae None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.666626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8761244-49f3-4b03-a134-545bcfa55dae None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.721641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599b1ee-06d6-4f6e-b389-f11910337f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.722233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599b1ee-06d6-4f6e-b389-f11910337f5e admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.722512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599b1ee-06d6-4f6e-b389-f11910337f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.722784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599b1ee-06d6-4f6e-b389-f11910337f5e admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.723404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599b1ee-06d6-4f6e-b389-f11910337f5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.725304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599b1ee-06d6-4f6e-b389-f11910337f5e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.746852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1299/2629] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:00:04 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:04.750887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641584f-1e7e-465b-b8e0-4de584d74cec admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.762367 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641584f-1e7e-465b-b8e0-4de584d74cec admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.766959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009245c-39f2-45a8-ab9d-3b91a4caee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.767551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009245c-39f2-45a8-ab9d-3b91a4caee5e admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.767831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009245c-39f2-45a8-ab9d-3b91a4caee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.768104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009245c-39f2-45a8-ab9d-3b91a4caee5e admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.768736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009245c-39f2-45a8-ab9d-3b91a4caee5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.770348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009245c-39f2-45a8-ab9d-3b91a4caee5e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.783647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1331/2630] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 02:00:04.786168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599b1ee-06d6-4f6e-b389-f11910337f5e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.796139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599b1ee-06d6-4f6e-b389-f11910337f5e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.800681 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31547290-bedc-4558-9c27-481cc7c2a0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.801349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31547290-bedc-4558-9c27-481cc7c2a0d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.801643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31547290-bedc-4558-9c27-481cc7c2a0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.801891 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31547290-bedc-4558-9c27-481cc7c2a0d6 admin admin] PATH_INFO: `/v3/projects/608b83becbad4daaa82f91336a4b051d/users/57f13a33f30a4193a526dd523e097b97/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.817181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31547290-bedc-4558-9c27-481cc7c2a0d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=608b83becbad4daaa82f91336a4b051d, user_id=57f13a33f30a4193a526dd523e097b97, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.819828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31547290-bedc-4558-9c27-481cc7c2a0d6 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.833821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1300/2631] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:00:04 2022] PUT /identity/v3/projects/608b83becbad4daaa82f91336a4b051d/users/57f13a33f30a4193a526dd523e097b97/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:04.837756 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009245c-39f2-45a8-ab9d-3b91a4caee5e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.847742 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009245c-39f2-45a8-ab9d-3b91a4caee5e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.907555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083394d3-8fbf-49d6-86ec-3f9b6dd6c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.908174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083394d3-8fbf-49d6-86ec-3f9b6dd6c7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.908452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083394d3-8fbf-49d6-86ec-3f9b6dd6c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.908726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083394d3-8fbf-49d6-86ec-3f9b6dd6c7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.909403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083394d3-8fbf-49d6-86ec-3f9b6dd6c7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.910993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083394d3-8fbf-49d6-86ec-3f9b6dd6c7c3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.922080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1332/2632] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:04.926125 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31547290-bedc-4558-9c27-481cc7c2a0d6 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:04.937271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31547290-bedc-4558-9c27-481cc7c2a0d6 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:04.941999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:04.942650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.942920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.943217 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] PATH_INFO: `/v3/projects/608b83becbad4daaa82f91336a4b051d/users/57f13a33f30a4193a526dd523e097b97/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:04.945131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=608b83becbad4daaa82f91336a4b051d, user_id=57f13a33f30a4193a526dd523e097b97, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:04.948361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:04.962128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1301/2633] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:00:04 2022] PUT /identity/v3/projects/608b83becbad4daaa82f91336a4b051d/users/57f13a33f30a4193a526dd523e097b97/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:00:04.972197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c5c7e-5169-4a0d-ad9d-ed22b952cf6b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:04.972479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c5c7e-5169-4a0d-ad9d-ed22b952cf6b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:04.972885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c5c7e-5169-4a0d-ad9d-ed22b952cf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:05.001006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-223c5c7e-5169-4a0d-ad9d-ed22b952cf6b None None] MFA Rules not processed for user `57f13a33f30a4193a526dd523e097b97`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:05.002184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-223c5c7e-5169-4a0d-ad9d-ed22b952cf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:05.068028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1333/2634] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:05.079105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:05.080774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:05.137332 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8cd2b-7c8d-4a11-9111-6d6a13b0b670 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:05.140889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cb54f-3173-4fe4-8e2c-3dfd5f55f120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:05.141506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb54f-3173-4fe4-8e2c-3dfd5f55f120 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:05.141780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb54f-3173-4fe4-8e2c-3dfd5f55f120 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:05.142045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cb54f-3173-4fe4-8e2c-3dfd5f55f120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:05.150689 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cb54f-3173-4fe4-8e2c-3dfd5f55f120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:05.152088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cb54f-3173-4fe4-8e2c-3dfd5f55f120 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:05.161184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1302/2635] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:05.295649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c5c7e-5169-4a0d-ad9d-ed22b952cf6b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:05.350870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:05.357593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c5c7e-5169-4a0d-ad9d-ed22b952cf6b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:05.362324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:05.362959 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:05.363244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:05.363522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] PATH_INFO: `/v3/users/aa1e2e4bc25148bc80e0aff5efcc9b55` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:05.364905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1e2e4bc25148bc80e0aff5efcc9b55)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:05.366877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:05.413384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1e2e4bc25148bc80e0aff5efcc9b55'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.414054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1e2e4bc25148bc80e0aff5efcc9b55'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.426423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1e2e4bc25148bc80e0aff5efcc9b55'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.433553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1e2e4bc25148bc80e0aff5efcc9b55'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.434157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1e2e4bc25148bc80e0aff5efcc9b55'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.447150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1334/2636] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:00:05 2022] DELETE /identity/v3/users/aa1e2e4bc25148bc80e0aff5efcc9b55 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:05.451542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f1f6c-9c67-432d-854f-5d45205dd095 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:05.451830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f1f6c-9c67-432d-854f-5d45205dd095 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:05.452104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f1f6c-9c67-432d-854f-5d45205dd095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:05.470871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d90f1f6c-9c67-432d-854f-5d45205dd095 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:05.472058 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d90f1f6c-9c67-432d-854f-5d45205dd095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:05.510092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:05.531379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1303/2637] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:05.541095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:05.542498 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:05.598411 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075d0a-5a58-4abd-8f54-ec27077be47e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:05.602076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504c6fe-b19b-4e72-a1d2-fd4b440d1ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:05.603285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504c6fe-b19b-4e72-a1d2-fd4b440d1ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:05.603565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504c6fe-b19b-4e72-a1d2-fd4b440d1ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:05.603838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504c6fe-b19b-4e72-a1d2-fd4b440d1ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:05.612682 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504c6fe-b19b-4e72-a1d2-fd4b440d1ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:05.614143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504c6fe-b19b-4e72-a1d2-fd4b440d1ad6 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:05.623655 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1335/2638] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:05.892118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f1f6c-9c67-432d-854f-5d45205dd095 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:05.893662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d90f1f6c-9c67-432d-854f-5d45205dd095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:05.906823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f1f6c-9c67-432d-854f-5d45205dd095 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:05.911706 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:05.912569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:05.912914 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:05.913278 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] PATH_INFO: `/v3/projects/6e6e12cd026c408197d5593df3d9363d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:05.914519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6e12cd026c408197d5593df3d9363d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:05.917350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:05.964234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6e12cd026c408197d5593df3d9363d', 'request_id': 'req-fa22e811-4785-4394-b32e-acf1a10abfaf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.970008 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6e12cd026c408197d5593df3d9363d', 'request_id': 'req-fa22e811-4785-4394-b32e-acf1a10abfaf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.976420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6e12cd026c408197d5593df3d9363d', 'request_id': 'req-fa22e811-4785-4394-b32e-acf1a10abfaf'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.990281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] The token cache is being invalidate because project 6e6e12cd026c408197d5593df3d9363d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:00:05.990571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.991196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-fa22e811-4785-4394-b32e-acf1a10abfaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:05.992676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1304/2639] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:00:05 2022] DELETE /identity/v3/projects/6e6e12cd026c408197d5593df3d9363d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:05.996793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504c6fe-b19b-4e72-a1d2-fd4b440d1ad6 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:06.006543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504c6fe-b19b-4e72-a1d2-fd4b440d1ad6 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:06.066145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:06.066778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.067056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.067325 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] PATH_INFO: `/v3/users/57f13a33f30a4193a526dd523e097b97` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.068440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57f13a33f30a4193a526dd523e097b97)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:06.070371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:06.092465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:06.121136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57f13a33f30a4193a526dd523e097b97'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.121823 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57f13a33f30a4193a526dd523e097b97'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.134232 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57f13a33f30a4193a526dd523e097b97'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.141725 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57f13a33f30a4193a526dd523e097b97'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.142341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57f13a33f30a4193a526dd523e097b97'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.154620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1336/2640] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:00:05 2022] DELETE /identity/v3/users/57f13a33f30a4193a526dd523e097b97 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:06.193379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8db7a-2308-427e-b317-cd9e038dc552 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.193772 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8db7a-2308-427e-b317-cd9e038dc552 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.194220 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8db7a-2308-427e-b317-cd9e038dc552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.216992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ebb8db7a-2308-427e-b317-cd9e038dc552 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:06.218347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ebb8db7a-2308-427e-b317-cd9e038dc552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:06.277270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1305/2641] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 02:00:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:06.291606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:06.293052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:06.373014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcd927-4ce4-4a1b-a461-901d4ce2d4d9 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:06.378698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83253c2-0662-4fa4-8991-37f0e276ee57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:06.379313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83253c2-0662-4fa4-8991-37f0e276ee57 service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.379593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83253c2-0662-4fa4-8991-37f0e276ee57 service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.379895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83253c2-0662-4fa4-8991-37f0e276ee57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.389544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83253c2-0662-4fa4-8991-37f0e276ee57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:06.391134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83253c2-0662-4fa4-8991-37f0e276ee57 service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:06.399869 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1337/2642] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 02:00:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 02:00:06.436403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8db7a-2308-427e-b317-cd9e038dc552 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:06.437803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ebb8db7a-2308-427e-b317-cd9e038dc552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:06.521228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8db7a-2308-427e-b317-cd9e038dc552 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:06.526080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:06.526739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.527016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.527287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] PATH_INFO: `/v3/projects/608b83becbad4daaa82f91336a4b051d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.528419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] RBAC: Authorizing `identity:delete_project(project_id=608b83becbad4daaa82f91336a4b051d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:06.530369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:06.542522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:06.584597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '608b83becbad4daaa82f91336a4b051d', 'request_id': 'req-76d63f44-4fa0-4556-a720-49248c36a2de'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.590576 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '608b83becbad4daaa82f91336a4b051d', 'request_id': 'req-76d63f44-4fa0-4556-a720-49248c36a2de'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.597932 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '608b83becbad4daaa82f91336a4b051d', 'request_id': 'req-76d63f44-4fa0-4556-a720-49248c36a2de'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.611459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] The token cache is being invalidate because project 608b83becbad4daaa82f91336a4b051d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:00:06.611747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.612364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-76d63f44-4fa0-4556-a720-49248c36a2de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:06.613879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1306/2643] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:00:06 2022] DELETE /identity/v3/projects/608b83becbad4daaa82f91336a4b051d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:06.619558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a8f2e-baed-4db2-8d18-ec8eacaf597b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.619992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a8f2e-baed-4db2-8d18-ec8eacaf597b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.620335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a8f2e-baed-4db2-8d18-ec8eacaf597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.648212 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-581a8f2e-baed-4db2-8d18-ec8eacaf597b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:06.650129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-581a8f2e-baed-4db2-8d18-ec8eacaf597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:06.714751 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1338/2644] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:06.722421 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e748fc-2cc3-4756-982d-438d781f1362 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.722700 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e748fc-2cc3-4756-982d-438d781f1362 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.723002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e748fc-2cc3-4756-982d-438d781f1362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.742820 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-08e748fc-2cc3-4756-982d-438d781f1362 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:06.744050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-08e748fc-2cc3-4756-982d-438d781f1362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:06.751847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1307/2645] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:06.756113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a8f2e-baed-4db2-8d18-ec8eacaf597b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:06.766257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a8f2e-baed-4db2-8d18-ec8eacaf597b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:06.770781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:06.771372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.771646 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.771914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.772548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:00:06.772843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:06.775087 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:06.786797 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1339/2646] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:00:06.791266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e748fc-2cc3-4756-982d-438d781f1362 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:06.801729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e748fc-2cc3-4756-982d-438d781f1362 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:06.806358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92136c58-6c73-4e25-9a06-7c1b181aa2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:06.806958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92136c58-6c73-4e25-9a06-7c1b181aa2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.807238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92136c58-6c73-4e25-9a06-7c1b181aa2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.807510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92136c58-6c73-4e25-9a06-7c1b181aa2c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.808148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92136c58-6c73-4e25-9a06-7c1b181aa2c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:06.810087 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92136c58-6c73-4e25-9a06-7c1b181aa2c8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:06.839238 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1308/2647] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:00:06 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 02:00:06.843391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:06.853272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408c589-7b98-4e20-9203-2b5ea27169ca admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:06.861922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:06.912100 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b650b84-fa71-4d41-8609-4ee43c7493d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:06.912712 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650b84-fa71-4d41-8609-4ee43c7493d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.912992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650b84-fa71-4d41-8609-4ee43c7493d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.913376 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b650b84-fa71-4d41-8609-4ee43c7493d4 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.914158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b650b84-fa71-4d41-8609-4ee43c7493d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:06.917039 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b650b84-fa71-4d41-8609-4ee43c7493d4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:06.947933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1340/2648] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:00:06 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:06.949263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92136c58-6c73-4e25-9a06-7c1b181aa2c8 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:06.960086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92136c58-6c73-4e25-9a06-7c1b181aa2c8 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:06.965201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b99d-952e-46c7-82b4-08d6c3c900f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:06.965848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b99d-952e-46c7-82b4-08d6c3c900f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:06.966131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b99d-952e-46c7-82b4-08d6c3c900f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:06.966406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b99d-952e-46c7-82b4-08d6c3c900f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:06.967111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791b99d-952e-46c7-82b4-08d6c3c900f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:06.968693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791b99d-952e-46c7-82b4-08d6c3c900f7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:06.979219 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1309/2649] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:06.983202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b650b84-fa71-4d41-8609-4ee43c7493d4 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:06.994912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b650b84-fa71-4d41-8609-4ee43c7493d4 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.000108 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7272fe-fed8-4885-a957-547181535d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.000876 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7272fe-fed8-4885-a957-547181535d20 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.001187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7272fe-fed8-4885-a957-547181535d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.001482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7272fe-fed8-4885-a957-547181535d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.002114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7272fe-fed8-4885-a957-547181535d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.003695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7272fe-fed8-4885-a957-547181535d20 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.015305 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1341/2650] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 02:00:07.019554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b99d-952e-46c7-82b4-08d6c3c900f7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.029477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b99d-952e-46c7-82b4-08d6c3c900f7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.034022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa0a3-aa6d-4781-9274-ce8e4a65a05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.034661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa0a3-aa6d-4781-9274-ce8e4a65a05c admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.034938 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa0a3-aa6d-4781-9274-ce8e4a65a05c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.035212 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa0a3-aa6d-4781-9274-ce8e4a65a05c admin admin] PATH_INFO: `/v3/projects/71f8bb8d3a5749c7b86439f5d1bc6f36/users/54aee97d1b0f49e78eee63f486b47208/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.049499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fa0a3-aa6d-4781-9274-ce8e4a65a05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f8bb8d3a5749c7b86439f5d1bc6f36, user_id=54aee97d1b0f49e78eee63f486b47208, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.052118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fa0a3-aa6d-4781-9274-ce8e4a65a05c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.065771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1310/2651] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:00:07 2022] PUT /identity/v3/projects/71f8bb8d3a5749c7b86439f5d1bc6f36/users/54aee97d1b0f49e78eee63f486b47208/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:07.077961 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0192977-0d7c-4095-b340-02b29168ba82 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.078249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0192977-0d7c-4095-b340-02b29168ba82 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.078548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0192977-0d7c-4095-b340-02b29168ba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.106042 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f0192977-0d7c-4095-b340-02b29168ba82 None None] MFA Rules not processed for user `54aee97d1b0f49e78eee63f486b47208`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:07.107295 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f0192977-0d7c-4095-b340-02b29168ba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:07.171150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1342/2652] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:07.175831 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa0a3-aa6d-4781-9274-ce8e4a65a05c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.240893 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa0a3-aa6d-4781-9274-ce8e4a65a05c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.247180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9010b4-2dca-466f-bb8e-1b4785e924c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.247785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9010b4-2dca-466f-bb8e-1b4785e924c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.248058 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9010b4-2dca-466f-bb8e-1b4785e924c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.248325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9010b4-2dca-466f-bb8e-1b4785e924c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.248944 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9010b4-2dca-466f-bb8e-1b4785e924c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.250848 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9010b4-2dca-466f-bb8e-1b4785e924c1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.284733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1311/2653] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:00:07 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:07.285884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0192977-0d7c-4095-b340-02b29168ba82 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.298753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0192977-0d7c-4095-b340-02b29168ba82 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.360374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462d4de-7e8a-43f2-b3b5-f3eb929f13ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.360900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462d4de-7e8a-43f2-b3b5-f3eb929f13ea admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.361218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462d4de-7e8a-43f2-b3b5-f3eb929f13ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.361496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462d4de-7e8a-43f2-b3b5-f3eb929f13ea admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.362112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462d4de-7e8a-43f2-b3b5-f3eb929f13ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.364013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462d4de-7e8a-43f2-b3b5-f3eb929f13ea admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.385361 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1343/2654] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:00:07 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:07.389657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9010b4-2dca-466f-bb8e-1b4785e924c1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.400671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9010b4-2dca-466f-bb8e-1b4785e924c1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.405233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48060d-76d1-473e-a0a4-31f788440890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.405949 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48060d-76d1-473e-a0a4-31f788440890 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.406233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48060d-76d1-473e-a0a4-31f788440890 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.406510 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48060d-76d1-473e-a0a4-31f788440890 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.407142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48060d-76d1-473e-a0a4-31f788440890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.408712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48060d-76d1-473e-a0a4-31f788440890 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.419054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1312/2655] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 02:00:07.422988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462d4de-7e8a-43f2-b3b5-f3eb929f13ea admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.432909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462d4de-7e8a-43f2-b3b5-f3eb929f13ea admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.437546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8d916-af82-4cdb-b8c0-d58165ce5b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.438463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d916-af82-4cdb-b8c0-d58165ce5b91 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.438749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d916-af82-4cdb-b8c0-d58165ce5b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.439023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8d916-af82-4cdb-b8c0-d58165ce5b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.440270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8d916-af82-4cdb-b8c0-d58165ce5b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.441887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8d916-af82-4cdb-b8c0-d58165ce5b91 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.447595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:07.452781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1344/2656] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:07.457092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48060d-76d1-473e-a0a4-31f788440890 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.467750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48060d-76d1-473e-a0a4-31f788440890 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.472269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922b50f-3e60-4976-9f33-3018fcda006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.472904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922b50f-3e60-4976-9f33-3018fcda006b admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.473210 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922b50f-3e60-4976-9f33-3018fcda006b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.473483 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922b50f-3e60-4976-9f33-3018fcda006b admin admin] PATH_INFO: `/v3/projects/9d69baa1a33d4231b90cf300132e8999/users/9a67f503817a473e8ae31746700366ff/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.481320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:07.489883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922b50f-3e60-4976-9f33-3018fcda006b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d69baa1a33d4231b90cf300132e8999, user_id=9a67f503817a473e8ae31746700366ff, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.493098 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922b50f-3e60-4976-9f33-3018fcda006b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.506132 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1313/2657] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:00:07 2022] PUT /identity/v3/projects/9d69baa1a33d4231b90cf300132e8999/users/9a67f503817a473e8ae31746700366ff/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:07.519677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21358de-23cf-4709-91e6-daccad85bb49 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.519991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21358de-23cf-4709-91e6-daccad85bb49 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.520265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21358de-23cf-4709-91e6-daccad85bb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.548445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f21358de-23cf-4709-91e6-daccad85bb49 None None] MFA Rules not processed for user `9a67f503817a473e8ae31746700366ff`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:07.549684 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f21358de-23cf-4709-91e6-daccad85bb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:07.613150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1345/2658] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:07.617993 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922b50f-3e60-4976-9f33-3018fcda006b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.677829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922b50f-3e60-4976-9f33-3018fcda006b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.682899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b262e-70a9-4c12-9e6f-7eea2725018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.683486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b262e-70a9-4c12-9e6f-7eea2725018a admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.683787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b262e-70a9-4c12-9e6f-7eea2725018a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.684054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b262e-70a9-4c12-9e6f-7eea2725018a admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.684676 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b262e-70a9-4c12-9e6f-7eea2725018a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.686601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b262e-70a9-4c12-9e6f-7eea2725018a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.716374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1314/2659] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:00:07 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:07.717813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21358de-23cf-4709-91e6-daccad85bb49 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.729395 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21358de-23cf-4709-91e6-daccad85bb49 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.783818 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7f52f-04a3-4c78-b3d3-3b891e66d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.784415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7f52f-04a3-4c78-b3d3-3b891e66d033 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.784692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7f52f-04a3-4c78-b3d3-3b891e66d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.784965 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7f52f-04a3-4c78-b3d3-3b891e66d033 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.785642 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7f52f-04a3-4c78-b3d3-3b891e66d033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.787532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7f52f-04a3-4c78-b3d3-3b891e66d033 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.807849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1346/2660] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:00:07 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:07.811995 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b262e-70a9-4c12-9e6f-7eea2725018a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.822806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b262e-70a9-4c12-9e6f-7eea2725018a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.827368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21f389-4958-46b6-b647-707f09114cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.827958 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21f389-4958-46b6-b647-707f09114cac admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.828235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21f389-4958-46b6-b647-707f09114cac admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.828505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21f389-4958-46b6-b647-707f09114cac admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.829160 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21f389-4958-46b6-b647-707f09114cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.830746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21f389-4958-46b6-b647-707f09114cac admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.840991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1315/2661] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:07.845008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7f52f-04a3-4c78-b3d3-3b891e66d033 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.854882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7f52f-04a3-4c78-b3d3-3b891e66d033 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.859415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c3fd-32e0-4a69-99cc-5de94ea81a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.860049 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c3fd-32e0-4a69-99cc-5de94ea81a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.860324 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c3fd-32e0-4a69-99cc-5de94ea81a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.860595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400c3fd-32e0-4a69-99cc-5de94ea81a4a admin admin] PATH_INFO: `/v3/projects/4cd44e2abfc24d048e6772318aac7009/users/8b971ee3160548f09e45d1cb4f313461/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.875680 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400c3fd-32e0-4a69-99cc-5de94ea81a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd44e2abfc24d048e6772318aac7009, user_id=8b971ee3160548f09e45d1cb4f313461, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.878330 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1400c3fd-32e0-4a69-99cc-5de94ea81a4a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.891683 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1347/2662] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:00:07 2022] PUT /identity/v3/projects/4cd44e2abfc24d048e6772318aac7009/users/8b971ee3160548f09e45d1cb4f313461/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:00:07.898738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21f389-4958-46b6-b647-707f09114cac admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.908699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21f389-4958-46b6-b647-707f09114cac admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.964665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f8854-97c1-4cc7-af52-5e609f08b134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.965330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f8854-97c1-4cc7-af52-5e609f08b134 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.965575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f8854-97c1-4cc7-af52-5e609f08b134 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.965845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f8854-97c1-4cc7-af52-5e609f08b134 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:07.966473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f8854-97c1-4cc7-af52-5e609f08b134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:07.968062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f8854-97c1-4cc7-af52-5e609f08b134 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:07.977471 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1316/2663] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:07.981612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c3fd-32e0-4a69-99cc-5de94ea81a4a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:07.991906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1400c3fd-32e0-4a69-99cc-5de94ea81a4a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:07.997036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:07.997707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:07.997989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:07.998264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] PATH_INFO: `/v3/projects/4cd44e2abfc24d048e6772318aac7009/users/8b971ee3160548f09e45d1cb4f313461/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:08.000163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd44e2abfc24d048e6772318aac7009, user_id=8b971ee3160548f09e45d1cb4f313461, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:08.002813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:08.013412 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1348/2664] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:00:07 2022] PUT /identity/v3/projects/4cd44e2abfc24d048e6772318aac7009/users/8b971ee3160548f09e45d1cb4f313461/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:00:08.024464 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecf003-9bc9-4994-895d-5167da051afd None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:08.024774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecf003-9bc9-4994-895d-5167da051afd None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:08.025045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecf003-9bc9-4994-895d-5167da051afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:08.051931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-22ecf003-9bc9-4994-895d-5167da051afd None None] MFA Rules not processed for user `8b971ee3160548f09e45d1cb4f313461`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:08.053150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-22ecf003-9bc9-4994-895d-5167da051afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:08.095622 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:08.120602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1317/2665] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:08.132485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:08.134241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:08.179257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:08.191438 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c408ab-83d1-43c6-aa09-4e7a9d7e396a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:08.195036 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71f9d-0778-4b7c-a2e5-caf539ed90d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:08.195620 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71f9d-0778-4b7c-a2e5-caf539ed90d1 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:08.195893 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71f9d-0778-4b7c-a2e5-caf539ed90d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:08.196686 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71f9d-0778-4b7c-a2e5-caf539ed90d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:08.259225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b71f9d-0778-4b7c-a2e5-caf539ed90d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:08.260639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b71f9d-0778-4b7c-a2e5-caf539ed90d1 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:08.269401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1349/2666] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:09.768941 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecf003-9bc9-4994-895d-5167da051afd None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:09.770353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-22ecf003-9bc9-4994-895d-5167da051afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:09.782513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecf003-9bc9-4994-895d-5167da051afd None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:09.785991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789eb30d-0837-41e9-b202-f4b4a3c962a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:09.786584 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb30d-0837-41e9-b202-f4b4a3c962a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:09.786862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb30d-0837-41e9-b202-f4b4a3c962a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:09.787129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789eb30d-0837-41e9-b202-f4b4a3c962a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:09.841599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789eb30d-0837-41e9-b202-f4b4a3c962a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:09.843037 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789eb30d-0837-41e9-b202-f4b4a3c962a1 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:09.851784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1318/2667] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:10.368433 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71f9d-0778-4b7c-a2e5-caf539ed90d1 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:10.377733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71f9d-0778-4b7c-a2e5-caf539ed90d1 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:10.381459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68982860-677e-4718-b5b3-9f80ea56276a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:10.382052 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68982860-677e-4718-b5b3-9f80ea56276a service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:10.382326 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68982860-677e-4718-b5b3-9f80ea56276a service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:10.382592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68982860-677e-4718-b5b3-9f80ea56276a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:10.392230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68982860-677e-4718-b5b3-9f80ea56276a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:10.393668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68982860-677e-4718-b5b3-9f80ea56276a service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:10.403511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1350/2668] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:14.254067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789eb30d-0837-41e9-b202-f4b4a3c962a1 service cinder] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:14.315327 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789eb30d-0837-41e9-b202-f4b4a3c962a1 service cinder] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:14.319841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:14.320455 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:14.320728 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:14.321525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] PATH_INFO: `/v3/users/54aee97d1b0f49e78eee63f486b47208` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:14.322619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54aee97d1b0f49e78eee63f486b47208)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:14.324513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:14.371551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54aee97d1b0f49e78eee63f486b47208'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:14.372191 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54aee97d1b0f49e78eee63f486b47208'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:14.379598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54aee97d1b0f49e78eee63f486b47208'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:14.392349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54aee97d1b0f49e78eee63f486b47208'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:14.399926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54aee97d1b0f49e78eee63f486b47208'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:14.402036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:14.406486 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1319/2669] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:00:14 2022] DELETE /identity/v3/users/54aee97d1b0f49e78eee63f486b47208 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:14.411393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2b9cc-2fc3-4e16-b080-5e85e4a9faf9 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:14.411873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2b9cc-2fc3-4e16-b080-5e85e4a9faf9 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:14.411972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2b9cc-2fc3-4e16-b080-5e85e4a9faf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:14.551551 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ebc2b9cc-2fc3-4e16-b080-5e85e4a9faf9 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:14.552698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ebc2b9cc-2fc3-4e16-b080-5e85e4a9faf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:14.610454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1351/2670] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:14.620872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:14.622278 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:14.676771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16c5a-75b1-4362-bfce-ae1b7a235ac5 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:14.680821 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4994e3-2ad3-4c5e-aa55-302b3099bfee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:14.681442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4994e3-2ad3-4c5e-aa55-302b3099bfee service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:14.681717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4994e3-2ad3-4c5e-aa55-302b3099bfee service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:14.681990 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4994e3-2ad3-4c5e-aa55-302b3099bfee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:14.691242 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4994e3-2ad3-4c5e-aa55-302b3099bfee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:14.692659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4994e3-2ad3-4c5e-aa55-302b3099bfee service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:14.702513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1320/2671] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:14.967377 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2b9cc-2fc3-4e16-b080-5e85e4a9faf9 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:14.968875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ebc2b9cc-2fc3-4e16-b080-5e85e4a9faf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:14.983911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2b9cc-2fc3-4e16-b080-5e85e4a9faf9 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:14.988691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:14.989360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:14.989895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:14.990167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] PATH_INFO: `/v3/projects/71f8bb8d3a5749c7b86439f5d1bc6f36` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:14.991414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f8bb8d3a5749c7b86439f5d1bc6f36)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:14.994624 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:15.039297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f8bb8d3a5749c7b86439f5d1bc6f36', 'request_id': 'req-634ee413-1784-42c9-afb6-0ffe6cf77377'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.046158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f8bb8d3a5749c7b86439f5d1bc6f36', 'request_id': 'req-634ee413-1784-42c9-afb6-0ffe6cf77377'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.052549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f8bb8d3a5749c7b86439f5d1bc6f36', 'request_id': 'req-634ee413-1784-42c9-afb6-0ffe6cf77377'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.065329 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] The token cache is being invalidate because project 71f8bb8d3a5749c7b86439f5d1bc6f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:00:15.065615 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.066239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.067747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1352/2672] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:00:14 2022] DELETE /identity/v3/projects/71f8bb8d3a5749c7b86439f5d1bc6f36 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:15.071720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4994e3-2ad3-4c5e-aa55-302b3099bfee service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:15.081301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4994e3-2ad3-4c5e-aa55-302b3099bfee service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:15.136077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:15.136619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:15.136897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:15.137202 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] PATH_INFO: `/v3/users/9a67f503817a473e8ae31746700366ff` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:15.138300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a67f503817a473e8ae31746700366ff)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:15.140196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:15.184265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a67f503817a473e8ae31746700366ff'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.184901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a67f503817a473e8ae31746700366ff'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.192239 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a67f503817a473e8ae31746700366ff'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.203987 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a67f503817a473e8ae31746700366ff'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.210627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a67f503817a473e8ae31746700366ff'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.216916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1321/2673] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:00:15 2022] DELETE /identity/v3/users/9a67f503817a473e8ae31746700366ff => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:15.484963 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:15.486470 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:15.507365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ee413-1784-42c9-afb6-0ffe6cf77377 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:15.511718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:15.566157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:15.566773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:15.567050 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:15.567322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] PATH_INFO: `/v3/projects/9d69baa1a33d4231b90cf300132e8999` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:15.568407 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d69baa1a33d4231b90cf300132e8999)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:15.570293 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:15.614818 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d69baa1a33d4231b90cf300132e8999', 'request_id': 'req-a36f1551-0cc4-4886-8731-fdebeaab639c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.621312 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d69baa1a33d4231b90cf300132e8999', 'request_id': 'req-a36f1551-0cc4-4886-8731-fdebeaab639c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.626647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d69baa1a33d4231b90cf300132e8999', 'request_id': 'req-a36f1551-0cc4-4886-8731-fdebeaab639c'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.638369 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] The token cache is being invalidate because project 9d69baa1a33d4231b90cf300132e8999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:00:15.638653 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.639261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.640722 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1353/2674] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:00:15 2022] DELETE /identity/v3/projects/9d69baa1a33d4231b90cf300132e8999 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:15.644674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:15.646073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:15.658559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0afc9-7309-41b9-977f-bfb85711e2ed admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:15.676710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:15.716228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:15.716844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:15.717152 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:15.717425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] PATH_INFO: `/v3/users/8b971ee3160548f09e45d1cb4f313461` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:15.718518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b971ee3160548f09e45d1cb4f313461)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:15.720419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:15.765880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b971ee3160548f09e45d1cb4f313461'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.766551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b971ee3160548f09e45d1cb4f313461'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.774260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b971ee3160548f09e45d1cb4f313461'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.785953 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b971ee3160548f09e45d1cb4f313461'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.793264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-333edaa3-96cd-48f0-9438-5527bf66b6eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b971ee3160548f09e45d1cb4f313461'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:15.799433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1322/2675] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:00:15 2022] DELETE /identity/v3/users/8b971ee3160548f09e45d1cb4f313461 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:16.070197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.072385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:16.086550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f1551-0cc4-4886-8731-fdebeaab639c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.149975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.150596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.150868 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.151133 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] PATH_INFO: `/v3/projects/4cd44e2abfc24d048e6772318aac7009` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.152218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd44e2abfc24d048e6772318aac7009)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.154121 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.192540 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:16.201017 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd44e2abfc24d048e6772318aac7009', 'request_id': 'req-72619caf-c27b-415c-bf68-5882c7ff226b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:16.207518 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd44e2abfc24d048e6772318aac7009', 'request_id': 'req-72619caf-c27b-415c-bf68-5882c7ff226b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:16.212989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd44e2abfc24d048e6772318aac7009', 'request_id': 'req-72619caf-c27b-415c-bf68-5882c7ff226b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:16.224709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] The token cache is being invalidate because project 4cd44e2abfc24d048e6772318aac7009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:00:16.224987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:16.225731 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-72619caf-c27b-415c-bf68-5882c7ff226b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:16.227274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1354/2676] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:00:16 2022] DELETE /identity/v3/projects/4cd44e2abfc24d048e6772318aac7009 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:16.232178 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37595de7-663d-4451-9653-16c380f313a4 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.232470 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37595de7-663d-4451-9653-16c380f313a4 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.232748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37595de7-663d-4451-9653-16c380f313a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.253751 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-37595de7-663d-4451-9653-16c380f313a4 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:16.254959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-37595de7-663d-4451-9653-16c380f313a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:16.314081 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1323/2677] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:16.322598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432cf2-d3fe-4f91-b118-a821659dce6b None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.322907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432cf2-d3fe-4f91-b118-a821659dce6b None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.323180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe432cf2-d3fe-4f91-b118-a821659dce6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.342587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-fe432cf2-d3fe-4f91-b118-a821659dce6b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:16.343766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-fe432cf2-d3fe-4f91-b118-a821659dce6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:16.351461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1355/2678] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:16.356233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37595de7-663d-4451-9653-16c380f313a4 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.366086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37595de7-663d-4451-9653-16c380f313a4 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.371233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.371818 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.372093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.372361 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.372991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:00:16.373317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.375070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.387761 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1324/2679] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:00:16.392113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe432cf2-d3fe-4f91-b118-a821659dce6b None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.401933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe432cf2-d3fe-4f91-b118-a821659dce6b None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.406567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1acd70-712e-4a2f-af21-e448a7441032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.407155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1acd70-712e-4a2f-af21-e448a7441032 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.407431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1acd70-712e-4a2f-af21-e448a7441032 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.407701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1acd70-712e-4a2f-af21-e448a7441032 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.408321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1acd70-712e-4a2f-af21-e448a7441032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.410224 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1acd70-712e-4a2f-af21-e448a7441032 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.436673 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1356/2680] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:00:16 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:16.441020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.455221 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0996d8-2710-413e-9acd-c5fa78cea98a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.488432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:16.520193 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33982a52-6c85-404d-9fb6-8f96d6a76dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.520792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33982a52-6c85-404d-9fb6-8f96d6a76dcc admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.521094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33982a52-6c85-404d-9fb6-8f96d6a76dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.521371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33982a52-6c85-404d-9fb6-8f96d6a76dcc admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.521996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33982a52-6c85-404d-9fb6-8f96d6a76dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.524502 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33982a52-6c85-404d-9fb6-8f96d6a76dcc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.544646 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1325/2681] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:00:16 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:00:16.548720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1acd70-712e-4a2f-af21-e448a7441032 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.559109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1acd70-712e-4a2f-af21-e448a7441032 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.563748 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963094f4-98f3-49d4-a83f-22f3f2a0bbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.564335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963094f4-98f3-49d4-a83f-22f3f2a0bbbf admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.564611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963094f4-98f3-49d4-a83f-22f3f2a0bbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.564878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963094f4-98f3-49d4-a83f-22f3f2a0bbbf admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.565567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963094f4-98f3-49d4-a83f-22f3f2a0bbbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.567131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963094f4-98f3-49d4-a83f-22f3f2a0bbbf admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.577578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1357/2682] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:16.581247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33982a52-6c85-404d-9fb6-8f96d6a76dcc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.591490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33982a52-6c85-404d-9fb6-8f96d6a76dcc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.596056 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b41493-859f-47c6-9937-5fe993c19d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.596642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b41493-859f-47c6-9937-5fe993c19d77 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.596919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b41493-859f-47c6-9937-5fe993c19d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.597228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b41493-859f-47c6-9937-5fe993c19d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.597861 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b41493-859f-47c6-9937-5fe993c19d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.599435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b41493-859f-47c6-9937-5fe993c19d77 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.610025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1326/2683] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:00:16.614000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963094f4-98f3-49d4-a83f-22f3f2a0bbbf admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.624070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963094f4-98f3-49d4-a83f-22f3f2a0bbbf admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.628554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.629226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.629504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.629776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] PATH_INFO: `/v3/projects/a28986f6661d44e688c871b262c925f5/users/1466cce5bdbf45438ee1d12da59d0912/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.644988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a28986f6661d44e688c871b262c925f5, user_id=1466cce5bdbf45438ee1d12da59d0912, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.647600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.660532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1358/2684] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:00:16 2022] PUT /identity/v3/projects/a28986f6661d44e688c871b262c925f5/users/1466cce5bdbf45438ee1d12da59d0912/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:00:16.672617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c7e53-a30e-497d-a968-9d353d04abe5 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.673138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c7e53-a30e-497d-a968-9d353d04abe5 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.673264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c7e53-a30e-497d-a968-9d353d04abe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.700039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-df9c7e53-a30e-497d-a968-9d353d04abe5 None None] MFA Rules not processed for user `1466cce5bdbf45438ee1d12da59d0912`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:16.701275 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-df9c7e53-a30e-497d-a968-9d353d04abe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:16.765354 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1327/2685] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:16.776531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.778233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:16.834286 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa544ab4-f51a-459a-8ec8-5bdfd8f0ca6f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.837947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b033c-cc50-4736-8ef6-8bd6b1183bcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.838542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b033c-cc50-4736-8ef6-8bd6b1183bcb service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.838817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b033c-cc50-4736-8ef6-8bd6b1183bcb service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.839085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b033c-cc50-4736-8ef6-8bd6b1183bcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.847165 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b033c-cc50-4736-8ef6-8bd6b1183bcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.848572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b033c-cc50-4736-8ef6-8bd6b1183bcb service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.857237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1359/2686] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:16.882414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c7e53-a30e-497d-a968-9d353d04abe5 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:16.943520 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c7e53-a30e-497d-a968-9d353d04abe5 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:16.948793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:16.949439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:16.949720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:16.949996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] PATH_INFO: `/v3/users/1466cce5bdbf45438ee1d12da59d0912` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:16.951119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1466cce5bdbf45438ee1d12da59d0912)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:16.953004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:16.992411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:17.000115 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1466cce5bdbf45438ee1d12da59d0912'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.000952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1466cce5bdbf45438ee1d12da59d0912'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.009533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1466cce5bdbf45438ee1d12da59d0912'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.021233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1466cce5bdbf45438ee1d12da59d0912'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.027847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1466cce5bdbf45438ee1d12da59d0912'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.033663 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1328/2687] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:00:16 2022] DELETE /identity/v3/users/1466cce5bdbf45438ee1d12da59d0912 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:17.038168 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df2e4-1038-41fd-a5d3-ae5a438d8a13 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:17.038453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df2e4-1038-41fd-a5d3-ae5a438d8a13 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:17.038728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43df2e4-1038-41fd-a5d3-ae5a438d8a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:17.058234 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d43df2e4-1038-41fd-a5d3-ae5a438d8a13 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:00:17.059415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d43df2e4-1038-41fd-a5d3-ae5a438d8a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:17.125900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1360/2688] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:00:17.135614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:17.137248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:17.197866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f02ee7-7166-4ef0-a028-c158608c4d8f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:17.201513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:17.202139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:17.202439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:17.202753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:17.211756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:17.213215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:17.222928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1329/2689] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:00:17.484694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43df2e4-1038-41fd-a5d3-ae5a438d8a13 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:17.486123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d43df2e4-1038-41fd-a5d3-ae5a438d8a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:17.502414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43df2e4-1038-41fd-a5d3-ae5a438d8a13 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:17.508825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:17.509500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:17.509758 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:17.510128 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] PATH_INFO: `/v3/projects/a28986f6661d44e688c871b262c925f5` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:17.511159 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a28986f6661d44e688c871b262c925f5)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:17.513038 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:17.514644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:17.560666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a28986f6661d44e688c871b262c925f5', 'request_id': 'req-6a336a56-bbab-4314-b707-54204b0e4e64'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.566747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a28986f6661d44e688c871b262c925f5', 'request_id': 'req-6a336a56-bbab-4314-b707-54204b0e4e64'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.572984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a28986f6661d44e688c871b262c925f5', 'request_id': 'req-6a336a56-bbab-4314-b707-54204b0e4e64'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.585141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] The token cache is being invalidate because project a28986f6661d44e688c871b262c925f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:00:17.585422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.586030 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:00:17.588005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1361/2690] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:00:17 2022] DELETE /identity/v3/projects/a28986f6661d44e688c871b262c925f5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:00:27.218741 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:27.582056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:00:59.341438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:00:59.343084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:59.436967 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3af0a6-6be0-4822-9c40-f6b5f6dc0ab5 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:00:59.528416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:00:59.529158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:00:59.529514 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:00:59.529865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:00:59.532317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:00:59.547719 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:00:59.549666 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:00:59.564092 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1330/2691] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:00:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:00.314374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:00.316347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:00.332962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a336a56-bbab-4314-b707-54204b0e4e64 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:00.338666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:00.339360 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:00.339710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:00.340056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:00.341902 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:00.357477 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:00.359172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:00.374170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1362/2692] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:06.610264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:06.611708 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:06.672601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6921aff-9845-469f-9b84-e1859f26edeb service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:06.677445 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:06.678076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:06.678901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:06.678901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:06.680548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:06.726261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:06.759337 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:06.761862 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:06.772863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1331/2693] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:10.369313 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=6 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:10.653881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:10.655788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:10.671184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea8c3f-af71-4043-b59a-993ae020f478 service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:10.676459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:10.677109 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:10.677384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:10.677652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] PATH_INFO: `/v3/users/04b569b91d1045dc8257c8aaa868842a` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:10.700545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b569b91d1045dc8257c8aaa868842a)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:10.702557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:10.751357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b569b91d1045dc8257c8aaa868842a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:10.752093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b569b91d1045dc8257c8aaa868842a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:10.765388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b569b91d1045dc8257c8aaa868842a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:10.773605 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b569b91d1045dc8257c8aaa868842a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:10.774550 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b058165f-c8f9-417b-ae4f-33edecc13c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b569b91d1045dc8257c8aaa868842a'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:10.787022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1363/2694] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:01:10 2022] DELETE /identity/v3/users/04b569b91d1045dc8257c8aaa868842a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:10.982228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:10.993406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d579003-9bd3-42bb-a047-e8e7b6bf5bce service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.061330 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.061957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.062229 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.062497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] PATH_INFO: `/v3/projects/37932fe56b864be6b57f92a76e92cd09` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.073567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=37932fe56b864be6b57f92a76e92cd09)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.075507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.127362 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37932fe56b864be6b57f92a76e92cd09', 'request_id': 'req-5793b7ec-270f-4cb2-9822-4738ffb1c62d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:11.133148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37932fe56b864be6b57f92a76e92cd09', 'request_id': 'req-5793b7ec-270f-4cb2-9822-4738ffb1c62d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:11.139779 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37932fe56b864be6b57f92a76e92cd09', 'request_id': 'req-5793b7ec-270f-4cb2-9822-4738ffb1c62d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:11.152196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] The token cache is being invalidate because project 37932fe56b864be6b57f92a76e92cd09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:01:11.152412 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:11.153054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-5793b7ec-270f-4cb2-9822-4738ffb1c62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:11.154598 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1332/2695] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:01:10 2022] DELETE /identity/v3/projects/37932fe56b864be6b57f92a76e92cd09 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:11.159432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472022-7977-450a-b990-f9e4d3d36960 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.159714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472022-7977-450a-b990-f9e4d3d36960 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.159989 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a472022-7977-450a-b990-f9e4d3d36960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.180137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5a472022-7977-450a-b990-f9e4d3d36960 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:11.181359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5a472022-7977-450a-b990-f9e4d3d36960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:11.240134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1364/2696] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:11.249836 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9e91-5433-4f85-9bc2-14d7e969eb77 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.250123 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9e91-5433-4f85-9bc2-14d7e969eb77 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.250436 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a9e91-5433-4f85-9bc2-14d7e969eb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.270763 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-218a9e91-5433-4f85-9bc2-14d7e969eb77 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:11.271952 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-218a9e91-5433-4f85-9bc2-14d7e969eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:11.280404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1333/2697] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:11.284995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a472022-7977-450a-b990-f9e4d3d36960 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.295062 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a472022-7977-450a-b990-f9e4d3d36960 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.300505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.301252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.301567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.301835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.302511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:01:11.302812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.304403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.316729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1365/2698] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:01:11.320901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a9e91-5433-4f85-9bc2-14d7e969eb77 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.331381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a9e91-5433-4f85-9bc2-14d7e969eb77 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.336665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20943912-3bcd-485b-9859-90606d304c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.337287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20943912-3bcd-485b-9859-90606d304c18 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.337597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20943912-3bcd-485b-9859-90606d304c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.337870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20943912-3bcd-485b-9859-90606d304c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.338585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20943912-3bcd-485b-9859-90606d304c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.340467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20943912-3bcd-485b-9859-90606d304c18 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.371609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1334/2699] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:01:11 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:01:11.375794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.387102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1214a-5e13-4416-adbf-f0c6012f140b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.431054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:11.444549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f524115-9ea4-415c-8632-b1cc6c2114cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.445785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f524115-9ea4-415c-8632-b1cc6c2114cf admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.446068 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f524115-9ea4-415c-8632-b1cc6c2114cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.446333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f524115-9ea4-415c-8632-b1cc6c2114cf admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.446969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f524115-9ea4-415c-8632-b1cc6c2114cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.448904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f524115-9ea4-415c-8632-b1cc6c2114cf admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.469105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1366/2700] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:01:11 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:01:11.473298 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20943912-3bcd-485b-9859-90606d304c18 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.484490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20943912-3bcd-485b-9859-90606d304c18 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.489306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95486980-886d-4842-8f47-b39ee8d1d673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.489899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95486980-886d-4842-8f47-b39ee8d1d673 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.490180 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95486980-886d-4842-8f47-b39ee8d1d673 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.490451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95486980-886d-4842-8f47-b39ee8d1d673 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.491089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95486980-886d-4842-8f47-b39ee8d1d673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.493205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95486980-886d-4842-8f47-b39ee8d1d673 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.503907 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1335/2701] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:01:11.508873 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f524115-9ea4-415c-8632-b1cc6c2114cf admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.524526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f524115-9ea4-415c-8632-b1cc6c2114cf admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.531821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d582cce-6b45-413a-abd2-d4d95af7cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.532840 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d582cce-6b45-413a-abd2-d4d95af7cf73 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.533164 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d582cce-6b45-413a-abd2-d4d95af7cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.533442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d582cce-6b45-413a-abd2-d4d95af7cf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.534348 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d582cce-6b45-413a-abd2-d4d95af7cf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.535958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d582cce-6b45-413a-abd2-d4d95af7cf73 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.549231 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1367/2702] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:01:11.553463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95486980-886d-4842-8f47-b39ee8d1d673 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.564243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95486980-886d-4842-8f47-b39ee8d1d673 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.568881 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c128a-70a7-4896-93e1-987a891ab11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.569571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c128a-70a7-4896-93e1-987a891ab11a admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.570498 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c128a-70a7-4896-93e1-987a891ab11a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.570799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c128a-70a7-4896-93e1-987a891ab11a admin admin] PATH_INFO: `/v3/projects/728fc07c84f2458caace78b7d225f716/users/5c8d5a351b9c482695116d742d6927ff/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.583909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:11.589228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c128a-70a7-4896-93e1-987a891ab11a admin admin] RBAC: Authorizing `identity:create_grant(project_id=728fc07c84f2458caace78b7d225f716, user_id=5c8d5a351b9c482695116d742d6927ff, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.592071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c128a-70a7-4896-93e1-987a891ab11a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.605908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1336/2703] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:01:11 2022] PUT /identity/v3/projects/728fc07c84f2458caace78b7d225f716/users/5c8d5a351b9c482695116d742d6927ff/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:01:11.617120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3837-e8aa-40f4-b4a3-cd3288f43641 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.617440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3837-e8aa-40f4-b4a3-cd3288f43641 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.617721 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa3837-e8aa-40f4-b4a3-cd3288f43641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.646270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-10fa3837-e8aa-40f4-b4a3-cd3288f43641 None None] MFA Rules not processed for user `5c8d5a351b9c482695116d742d6927ff`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:11.647592 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-10fa3837-e8aa-40f4-b4a3-cd3288f43641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:11.712440 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1368/2704] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:11.716964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c128a-70a7-4896-93e1-987a891ab11a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.778222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c128a-70a7-4896-93e1-987a891ab11a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.782853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830dc7f-fe82-42c5-8382-ea7392b23b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.783444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830dc7f-fe82-42c5-8382-ea7392b23b29 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.783724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830dc7f-fe82-42c5-8382-ea7392b23b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.783997 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830dc7f-fe82-42c5-8382-ea7392b23b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.784611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830dc7f-fe82-42c5-8382-ea7392b23b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.786530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830dc7f-fe82-42c5-8382-ea7392b23b29 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.813159 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1337/2705] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:01:11 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:01:11.817406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa3837-e8aa-40f4-b4a3-cd3288f43641 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.828222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa3837-e8aa-40f4-b4a3-cd3288f43641 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.884203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c2beb-4862-4b2c-81a5-217b4b09b607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.884803 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c2beb-4862-4b2c-81a5-217b4b09b607 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.885098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c2beb-4862-4b2c-81a5-217b4b09b607 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.885373 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c2beb-4862-4b2c-81a5-217b4b09b607 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.886005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c2beb-4862-4b2c-81a5-217b4b09b607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.887883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c2beb-4862-4b2c-81a5-217b4b09b607 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.909880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1369/2706] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:01:11 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:01:11.914388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830dc7f-fe82-42c5-8382-ea7392b23b29 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.924849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830dc7f-fe82-42c5-8382-ea7392b23b29 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.930886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342db29-27b4-48b5-8ad8-287c4a5bc137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.931480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342db29-27b4-48b5-8ad8-287c4a5bc137 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.931785 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342db29-27b4-48b5-8ad8-287c4a5bc137 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.932056 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342db29-27b4-48b5-8ad8-287c4a5bc137 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.932727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342db29-27b4-48b5-8ad8-287c4a5bc137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.934379 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342db29-27b4-48b5-8ad8-287c4a5bc137 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.945005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1338/2707] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:01:11.949543 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c2beb-4862-4b2c-81a5-217b4b09b607 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:11.959406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c2beb-4862-4b2c-81a5-217b4b09b607 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:11.965701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ee7c4-95d1-4f09-af58-16fa4d72678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:11.966285 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ee7c4-95d1-4f09-af58-16fa4d72678d admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:11.966567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ee7c4-95d1-4f09-af58-16fa4d72678d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:11.966835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ee7c4-95d1-4f09-af58-16fa4d72678d admin admin] PATH_INFO: `/v3/projects/704033b9b0234c60b900c05a7e3b1069/users/405008feb6d1455aadea346488dc0083/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:11.981858 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ee7c4-95d1-4f09-af58-16fa4d72678d admin admin] RBAC: Authorizing `identity:create_grant(project_id=704033b9b0234c60b900c05a7e3b1069, user_id=405008feb6d1455aadea346488dc0083, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:11.984457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ee7c4-95d1-4f09-af58-16fa4d72678d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:11.997648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1370/2708] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:01:11 2022] PUT /identity/v3/projects/704033b9b0234c60b900c05a7e3b1069/users/405008feb6d1455aadea346488dc0083/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:12.001704 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342db29-27b4-48b5-8ad8-287c4a5bc137 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:12.016642 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342db29-27b4-48b5-8ad8-287c4a5bc137 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:12.071446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9c653-2578-4630-87e7-eeda52cfe286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:12.072050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c653-2578-4630-87e7-eeda52cfe286 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:12.072329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c653-2578-4630-87e7-eeda52cfe286 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:12.072599 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9c653-2578-4630-87e7-eeda52cfe286 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:12.073263 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9c653-2578-4630-87e7-eeda52cfe286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:12.074835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9c653-2578-4630-87e7-eeda52cfe286 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:12.084218 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1339/2709] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:01:12.089098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ee7c4-95d1-4f09-af58-16fa4d72678d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:12.099184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ee7c4-95d1-4f09-af58-16fa4d72678d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:12.103776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:12.104410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:12.104681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:12.104946 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] PATH_INFO: `/v3/projects/704033b9b0234c60b900c05a7e3b1069/users/405008feb6d1455aadea346488dc0083/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:12.106849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704033b9b0234c60b900c05a7e3b1069, user_id=405008feb6d1455aadea346488dc0083, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:12.109452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:12.119569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1371/2710] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:01:12 2022] PUT /identity/v3/projects/704033b9b0234c60b900c05a7e3b1069/users/405008feb6d1455aadea346488dc0083/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:12.130830 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8306a75-4960-432b-9f7a-b64ededa6a58 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:12.131118 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8306a75-4960-432b-9f7a-b64ededa6a58 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:12.131394 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8306a75-4960-432b-9f7a-b64ededa6a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:12.160481 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-a8306a75-4960-432b-9f7a-b64ededa6a58 None None] MFA Rules not processed for user `405008feb6d1455aadea346488dc0083`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:12.161692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a8306a75-4960-432b-9f7a-b64ededa6a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:12.209895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:12.231833 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1340/2711] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:12.243504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:12.245336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:12.255980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:12.312487 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e508319-e662-45cb-9568-85cd677b2f11 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:12.316182 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b447b-10a5-4eae-b610-4752c8ea71d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:12.316796 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b447b-10a5-4eae-b610-4752c8ea71d3 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:12.317088 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b447b-10a5-4eae-b610-4752c8ea71d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:12.317374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b447b-10a5-4eae-b610-4752c8ea71d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:12.327340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b447b-10a5-4eae-b610-4752c8ea71d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:12.328764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b447b-10a5-4eae-b610-4752c8ea71d3 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:12.340058 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1372/2712] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:12.390261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8306a75-4960-432b-9f7a-b64ededa6a58 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:12.391671 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a8306a75-4960-432b-9f7a-b64ededa6a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:12.424410 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b447b-10a5-4eae-b610-4752c8ea71d3 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:12.433619 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b447b-10a5-4eae-b610-4752c8ea71d3 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:12.437454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:12.438071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:12.438345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:12.438643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:12.453477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8306a75-4960-432b-9f7a-b64ededa6a58 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:12.457223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:12.457834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:12.458113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:12.458393 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:12.460112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:12.503900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:12.505258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:12.514768 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1373/2713] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 02:01:12.528798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:12.530300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:12.540694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1341/2714] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:19.120842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:19.122374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:19.137987 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50935-ecb8-4b2f-a7e2-65dda9e1b06e service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:19.143391 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:19.144043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:19.144338 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:19.144636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] PATH_INFO: `/v3/users/fa5465b818f34778a1d51278653488ed` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:19.160848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5465b818f34778a1d51278653488ed)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:19.162881 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:19.211317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5465b818f34778a1d51278653488ed'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.211975 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5465b818f34778a1d51278653488ed'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.223896 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5465b818f34778a1d51278653488ed'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.231249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5465b818f34778a1d51278653488ed'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.231900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5465b818f34778a1d51278653488ed'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.244829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1374/2715] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:01:19 2022] DELETE /identity/v3/users/fa5465b818f34778a1d51278653488ed => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:19.425892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:19.436306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c998235-b7ce-4d52-83f9-e3fd27f8f7dd service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:19.499744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:19.500358 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:19.500665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:19.500974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] PATH_INFO: `/v3/projects/826001ffb29b4d42a5c299154c17b93e` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:19.514462 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] RBAC: Authorizing `identity:delete_project(project_id=826001ffb29b4d42a5c299154c17b93e)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:19.516406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:19.555013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:19.569432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826001ffb29b4d42a5c299154c17b93e', 'request_id': 'req-53ece174-61f5-43f3-9222-ca8eef20804d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.575519 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826001ffb29b4d42a5c299154c17b93e', 'request_id': 'req-53ece174-61f5-43f3-9222-ca8eef20804d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.585559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826001ffb29b4d42a5c299154c17b93e', 'request_id': 'req-53ece174-61f5-43f3-9222-ca8eef20804d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.602537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] The token cache is being invalidate because project 826001ffb29b4d42a5c299154c17b93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:01:19.602935 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.603727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.605482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1342/2716] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:01:19 2022] DELETE /identity/v3/projects/826001ffb29b4d42a5c299154c17b93e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:19.609511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:19.611623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:19.628267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc3e9b-9553-458c-9223-6d6ac82d48db admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:19.638648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:19.689345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:19.689971 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:19.690245 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:19.690515 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] PATH_INFO: `/v3/users/4cf02609cda94190832e124ac3dbd043` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:19.703453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf02609cda94190832e124ac3dbd043)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:19.705743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:19.755514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf02609cda94190832e124ac3dbd043'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.756186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf02609cda94190832e124ac3dbd043'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.768283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf02609cda94190832e124ac3dbd043'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.776437 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf02609cda94190832e124ac3dbd043'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.777189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-ca50f2a0-98e5-41f6-8f77-eddb7243141b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf02609cda94190832e124ac3dbd043'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:19.792650 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1375/2717] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:01:19 2022] DELETE /identity/v3/users/4cf02609cda94190832e124ac3dbd043 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:19.974657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:19.976197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:19.989901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ece174-61f5-43f3-9222-ca8eef20804d admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:20.062811 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:20.063444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:20.063726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:20.063994 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] PATH_INFO: `/v3/projects/bd10498303de47c69683c94f24b83f18` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:20.074748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd10498303de47c69683c94f24b83f18)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:20.076634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:20.124886 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd10498303de47c69683c94f24b83f18', 'request_id': 'req-c7ea7719-ab6b-40b8-8265-f22d44cb6582'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:20.131234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd10498303de47c69683c94f24b83f18', 'request_id': 'req-c7ea7719-ab6b-40b8-8265-f22d44cb6582'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:20.138523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd10498303de47c69683c94f24b83f18', 'request_id': 'req-c7ea7719-ab6b-40b8-8265-f22d44cb6582'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:20.150206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] The token cache is being invalidate because project bd10498303de47c69683c94f24b83f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:01:20.150482 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:20.151097 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:20.152573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1343/2718] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:01:19 2022] DELETE /identity/v3/projects/bd10498303de47c69683c94f24b83f18 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:26.123101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c4dac-bf97-4abf-ad4f-5e3c7660b184 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:26.123579 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c4dac-bf97-4abf-ad4f-5e3c7660b184 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:26.123710 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c4dac-bf97-4abf-ad4f-5e3c7660b184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:26.142191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c45c4dac-bf97-4abf-ad4f-5e3c7660b184 None None] MFA Rules not processed for user `eaaf03c8c05d4f598328ef5c05974e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:26.143380 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c45c4dac-bf97-4abf-ad4f-5e3c7660b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:26.196520 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1376/2719] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 02:01:26.496372 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:26.497745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:26.550437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:26.570713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea7719-ab6b-40b8-8265-f22d44cb6582 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:26.574408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:26.575019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:26.575296 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:26.575718 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:26.577525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:26.590478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:26.592042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:26.601281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1344/2720] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:36.190007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:36.596754 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=8 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:38.693904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c4dac-bf97-4abf-ad4f-5e3c7660b184 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:38.695930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c45c4dac-bf97-4abf-ad4f-5e3c7660b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:38.767342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c4dac-bf97-4abf-ad4f-5e3c7660b184 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:38.772711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:38.773335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:38.773613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:38.773879 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:38.775553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:38.785536 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:38.787076 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:38.796169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1377/2721] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:44.795104 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:44.796570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:44.876946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a430a1-b6de-4955-a559-fe2cb24746b6 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:44.882733 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71f4ee-0bec-41e0-892c-ac3f1ca3a6fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:44.883349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f4ee-0bec-41e0-892c-ac3f1ca3a6fa service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:44.883743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f4ee-0bec-41e0-892c-ac3f1ca3a6fa service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:44.884034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71f4ee-0bec-41e0-892c-ac3f1ca3a6fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:44.886010 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0d71f4ee-0bec-41e0-892c-ac3f1ca3a6fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:44.901940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71f4ee-0bec-41e0-892c-ac3f1ca3a6fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:44.903553 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71f4ee-0bec-41e0-892c-ac3f1ca3a6fa service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:44.914659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1345/2722] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:48.792209 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=34 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:54.909698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=32 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:57.724737 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:57.726820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:57.788599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a84923-8746-4f17-a942-c9c5076122bd service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:57.792958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:57.794198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:57.794198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:57.794198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] PATH_INFO: `/v3/users/5c8d5a351b9c482695116d742d6927ff` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:57.796067 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c8d5a351b9c482695116d742d6927ff)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:57.798059 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:57.843623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c8d5a351b9c482695116d742d6927ff'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:57.844274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c8d5a351b9c482695116d742d6927ff'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:57.855813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c8d5a351b9c482695116d742d6927ff'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:57.863060 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c8d5a351b9c482695116d742d6927ff'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:57.864021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c8d5a351b9c482695116d742d6927ff'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:57.876240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1378/2723] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:01:57 2022] DELETE /identity/v3/users/5c8d5a351b9c482695116d742d6927ff => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:57.880665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f3989-c550-41c9-9907-4ee1eaff85c0 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:57.881771 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f3989-c550-41c9-9907-4ee1eaff85c0 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:57.881885 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f3989-c550-41c9-9907-4ee1eaff85c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:57.900406 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6b9f3989-c550-41c9-9907-4ee1eaff85c0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:57.901648 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6b9f3989-c550-41c9-9907-4ee1eaff85c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:57.959813 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1346/2724] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:57.969349 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:57.970715 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:58.028404 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d3a38-cedd-45c6-884e-333d3215ff8a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:58.031982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996f3f4-3f94-4919-b88e-05951d57dbed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:58.032594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996f3f4-3f94-4919-b88e-05951d57dbed service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:58.032849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996f3f4-3f94-4919-b88e-05951d57dbed service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:58.033150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996f3f4-3f94-4919-b88e-05951d57dbed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:58.041951 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996f3f4-3f94-4919-b88e-05951d57dbed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:58.043339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996f3f4-3f94-4919-b88e-05951d57dbed service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:58.052726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1379/2725] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:58.320682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f3989-c550-41c9-9907-4ee1eaff85c0 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:58.322280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6b9f3989-c550-41c9-9907-4ee1eaff85c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:58.335034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f3989-c550-41c9-9907-4ee1eaff85c0 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:58.339629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:58.340261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:58.340537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:58.340808 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] PATH_INFO: `/v3/projects/728fc07c84f2458caace78b7d225f716` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:58.342002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728fc07c84f2458caace78b7d225f716)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:58.343916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:58.395174 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728fc07c84f2458caace78b7d225f716', 'request_id': 'req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.400898 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728fc07c84f2458caace78b7d225f716', 'request_id': 'req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.407033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728fc07c84f2458caace78b7d225f716', 'request_id': 'req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.420047 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] The token cache is being invalidate because project 728fc07c84f2458caace78b7d225f716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:01:58.420333 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.420950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.422459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1347/2726] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:01:58 2022] DELETE /identity/v3/projects/728fc07c84f2458caace78b7d225f716 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:58.426441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996f3f4-3f94-4919-b88e-05951d57dbed service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:58.436598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996f3f4-3f94-4919-b88e-05951d57dbed service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:58.454073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:58.494319 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:58.494943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:58.495217 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:58.495485 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] PATH_INFO: `/v3/users/405008feb6d1455aadea346488dc0083` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:58.496593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405008feb6d1455aadea346488dc0083)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:58.498526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:58.542749 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405008feb6d1455aadea346488dc0083'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.543384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405008feb6d1455aadea346488dc0083'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.554982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405008feb6d1455aadea346488dc0083'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.561857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405008feb6d1455aadea346488dc0083'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.562464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3fafb335-13f3-4b33-92ca-bab25bf43495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405008feb6d1455aadea346488dc0083'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.574762 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1380/2727] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:01:58 2022] DELETE /identity/v3/users/405008feb6d1455aadea346488dc0083 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:58.849535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:58.851744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:58.865746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2e3f-0ef7-47e5-a02f-bf69cf1e74f9 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:58.923053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:58.923710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:58.923868 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:58.924142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] PATH_INFO: `/v3/projects/704033b9b0234c60b900c05a7e3b1069` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:58.925272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=704033b9b0234c60b900c05a7e3b1069)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:58.927170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:58.956662 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:58.973823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704033b9b0234c60b900c05a7e3b1069', 'request_id': 'req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.979835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704033b9b0234c60b900c05a7e3b1069', 'request_id': 'req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.985923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704033b9b0234c60b900c05a7e3b1069', 'request_id': 'req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.998156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] The token cache is being invalidate because project 704033b9b0234c60b900c05a7e3b1069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:01:58.998442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:58.999040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d4645ca2-c91f-4bce-96cc-17aae0bfa23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:59.000497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1348/2728] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:01:58 2022] DELETE /identity/v3/projects/704033b9b0234c60b900c05a7e3b1069 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:59.007366 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c325-3fb5-4b94-a671-d420ad622d67 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.007854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c325-3fb5-4b94-a671-d420ad622d67 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.007974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9c325-3fb5-4b94-a671-d420ad622d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.027819 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-6fb9c325-3fb5-4b94-a671-d420ad622d67 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:59.029000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6fb9c325-3fb5-4b94-a671-d420ad622d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:59.088742 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1381/2729] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:59.096875 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7fb88-3080-4660-84fb-1a676f078845 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.097200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7fb88-3080-4660-84fb-1a676f078845 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.097475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb7fb88-3080-4660-84fb-1a676f078845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.118368 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-feb7fb88-3080-4660-84fb-1a676f078845 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:59.119561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-feb7fb88-3080-4660-84fb-1a676f078845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:59.127411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1349/2730] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:59.131500 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9c325-3fb5-4b94-a671-d420ad622d67 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.141146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9c325-3fb5-4b94-a671-d420ad622d67 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.145648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.146236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.146514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.146812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.147442 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:01:59.147740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.149354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.160609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1382/2731] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:01:59.165236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7fb88-3080-4660-84fb-1a676f078845 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.174278 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb7fb88-3080-4660-84fb-1a676f078845 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.179435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658abf75-50ae-4c4c-99c9-eb812025e353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.180020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abf75-50ae-4c4c-99c9-eb812025e353 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.180294 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abf75-50ae-4c4c-99c9-eb812025e353 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.180564 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658abf75-50ae-4c4c-99c9-eb812025e353 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.181217 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658abf75-50ae-4c4c-99c9-eb812025e353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.183073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658abf75-50ae-4c4c-99c9-eb812025e353 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.210260 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1350/2732] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:01:59 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:01:59.214253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.224001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5225db-d2a7-4846-8422-d186482149d1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.250114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:59.280698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639f5cf-1a5f-4713-988a-ae6db286637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.281357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639f5cf-1a5f-4713-988a-ae6db286637a admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.281638 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639f5cf-1a5f-4713-988a-ae6db286637a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.281908 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639f5cf-1a5f-4713-988a-ae6db286637a admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.282532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639f5cf-1a5f-4713-988a-ae6db286637a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.284453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639f5cf-1a5f-4713-988a-ae6db286637a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.306070 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1383/2733] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:01:59 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 02:01:59.310170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658abf75-50ae-4c4c-99c9-eb812025e353 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.320293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658abf75-50ae-4c4c-99c9-eb812025e353 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.324826 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4902b6-a8d1-4f41-87de-8e086fc216e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.325440 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4902b6-a8d1-4f41-87de-8e086fc216e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.325717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4902b6-a8d1-4f41-87de-8e086fc216e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.325984 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4902b6-a8d1-4f41-87de-8e086fc216e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.326618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4902b6-a8d1-4f41-87de-8e086fc216e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.328246 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4902b6-a8d1-4f41-87de-8e086fc216e7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.338042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1351/2734] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:01:59.341787 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639f5cf-1a5f-4713-988a-ae6db286637a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.352044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639f5cf-1a5f-4713-988a-ae6db286637a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.356558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba6ab0-08c0-4e09-837e-4eb17b8dfd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.357172 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba6ab0-08c0-4e09-837e-4eb17b8dfd9d admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.357452 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba6ab0-08c0-4e09-837e-4eb17b8dfd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.357720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba6ab0-08c0-4e09-837e-4eb17b8dfd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.358342 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba6ab0-08c0-4e09-837e-4eb17b8dfd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.359889 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba6ab0-08c0-4e09-837e-4eb17b8dfd9d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.371043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1384/2735] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:01:59.374903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4902b6-a8d1-4f41-87de-8e086fc216e7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.386247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4902b6-a8d1-4f41-87de-8e086fc216e7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.391535 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.392179 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.392456 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.392726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] PATH_INFO: `/v3/projects/fd964bc23f4c4285a191f31da9711f04/users/9e0c7d2bbbc6469cb7cec2bcbdbf7bcd/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.407695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd964bc23f4c4285a191f31da9711f04, user_id=9e0c7d2bbbc6469cb7cec2bcbdbf7bcd, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.410357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.423940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1352/2736] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:01:59 2022] PUT /identity/v3/projects/fd964bc23f4c4285a191f31da9711f04/users/9e0c7d2bbbc6469cb7cec2bcbdbf7bcd/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:59.435878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321ea11-b962-408b-91e8-9da97053002e None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.436092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321ea11-b962-408b-91e8-9da97053002e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.436359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321ea11-b962-408b-91e8-9da97053002e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.467314 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4321ea11-b962-408b-91e8-9da97053002e None None] MFA Rules not processed for user `9e0c7d2bbbc6469cb7cec2bcbdbf7bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:59.468493 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4321ea11-b962-408b-91e8-9da97053002e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:59.531081 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1385/2737] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:59.542043 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.543746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:59.599760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e63be-0a8a-4ba7-94a8-77ca366c4ac7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.603432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79f5e7-cfa1-4f67-9e54-973ff7ed6b2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.604029 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f5e7-cfa1-4f67-9e54-973ff7ed6b2e service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.604303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f5e7-cfa1-4f67-9e54-973ff7ed6b2e service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.604569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79f5e7-cfa1-4f67-9e54-973ff7ed6b2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.612588 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79f5e7-cfa1-4f67-9e54-973ff7ed6b2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.614288 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79f5e7-cfa1-4f67-9e54-973ff7ed6b2e service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.623345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1353/2738] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:01:59.642958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321ea11-b962-408b-91e8-9da97053002e None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.706549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321ea11-b962-408b-91e8-9da97053002e None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.711886 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.712559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.712863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.713188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] PATH_INFO: `/v3/users/9e0c7d2bbbc6469cb7cec2bcbdbf7bcd` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.714393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0c7d2bbbc6469cb7cec2bcbdbf7bcd)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.716496 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.758571 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:59.765163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0c7d2bbbc6469cb7cec2bcbdbf7bcd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:59.766221 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0c7d2bbbc6469cb7cec2bcbdbf7bcd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:59.778707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0c7d2bbbc6469cb7cec2bcbdbf7bcd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:59.786136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0c7d2bbbc6469cb7cec2bcbdbf7bcd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:59.786857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0c7d2bbbc6469cb7cec2bcbdbf7bcd'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:01:59.798978 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1386/2739] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:01:59 2022] DELETE /identity/v3/users/9e0c7d2bbbc6469cb7cec2bcbdbf7bcd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:01:59.803713 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22993bbd-c343-4a66-91b2-a70895a6cb21 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.804060 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22993bbd-c343-4a66-91b2-a70895a6cb21 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.804352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22993bbd-c343-4a66-91b2-a70895a6cb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.825185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-22993bbd-c343-4a66-91b2-a70895a6cb21 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:01:59.826477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-22993bbd-c343-4a66-91b2-a70895a6cb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:59.874526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:01:59.889743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1354/2740] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:01:59.900061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:01:59.901488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:01:59.958859 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f1594-098a-4a4c-933b-8c47f5dc7717 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:01:59.962419 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa9f0c-5cd3-41a5-a3de-c2e22e616619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:01:59.963019 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa9f0c-5cd3-41a5-a3de-c2e22e616619 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:01:59.963291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa9f0c-5cd3-41a5-a3de-c2e22e616619 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:01:59.963557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa9f0c-5cd3-41a5-a3de-c2e22e616619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:01:59.972141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa9f0c-5cd3-41a5-a3de-c2e22e616619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:01:59.973599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa9f0c-5cd3-41a5-a3de-c2e22e616619 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:01:59.983253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1387/2741] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 02:02:00.251550 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22993bbd-c343-4a66-91b2-a70895a6cb21 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:00.254150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-22993bbd-c343-4a66-91b2-a70895a6cb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:00.269512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22993bbd-c343-4a66-91b2-a70895a6cb21 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:00.274915 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:00.275554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.275834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.276135 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] PATH_INFO: `/v3/projects/fd964bc23f4c4285a191f31da9711f04` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.277369 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd964bc23f4c4285a191f31da9711f04)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:00.279156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:00.327892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd964bc23f4c4285a191f31da9711f04', 'request_id': 'req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:00.333279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd964bc23f4c4285a191f31da9711f04', 'request_id': 'req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:00.339739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd964bc23f4c4285a191f31da9711f04', 'request_id': 'req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:00.352605 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] The token cache is being invalidate because project fd964bc23f4c4285a191f31da9711f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:02:00.353023 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:00.353819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-54a3d0a4-ce37-457e-be4e-f8d5db5ac59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:00.355611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1355/2742] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:02:00 2022] DELETE /identity/v3/projects/fd964bc23f4c4285a191f31da9711f04 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:00.360754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc4428-4c41-4900-a50d-9d9f78bf2202 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.361252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc4428-4c41-4900-a50d-9d9f78bf2202 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.361351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc4428-4c41-4900-a50d-9d9f78bf2202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.380839 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-18cc4428-4c41-4900-a50d-9d9f78bf2202 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:00.382082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-18cc4428-4c41-4900-a50d-9d9f78bf2202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:00.444317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1388/2743] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:00.452655 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4616b-35f6-4b88-872a-e08047d390c1 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.452940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4616b-35f6-4b88-872a-e08047d390c1 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.453247 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4616b-35f6-4b88-872a-e08047d390c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.474295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8ca4616b-35f6-4b88-872a-e08047d390c1 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:00.476195 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8ca4616b-35f6-4b88-872a-e08047d390c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:00.484568 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1356/2744] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:00.489829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc4428-4c41-4900-a50d-9d9f78bf2202 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:00.500528 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc4428-4c41-4900-a50d-9d9f78bf2202 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:00.505403 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:00.506594 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.506899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.507201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.507899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:02:00.508240 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:00.510025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:00.522730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1389/2745] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:02:00.526809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4616b-35f6-4b88-872a-e08047d390c1 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:00.535991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4616b-35f6-4b88-872a-e08047d390c1 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:00.540593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56605351-9fa6-4766-9be5-ca15e5b0b66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:00.541215 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605351-9fa6-4766-9be5-ca15e5b0b66a admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.541493 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605351-9fa6-4766-9be5-ca15e5b0b66a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.541760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56605351-9fa6-4766-9be5-ca15e5b0b66a admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.542385 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56605351-9fa6-4766-9be5-ca15e5b0b66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:00.544262 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56605351-9fa6-4766-9be5-ca15e5b0b66a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:00.571079 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1357/2746] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:02:00 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:00.575156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:00.585691 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e12345-dfb0-4bc7-9432-98331f58fb0a admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:00.627589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:00.641714 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11247dab-0716-43b5-941a-6ec624dea5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:00.642308 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247dab-0716-43b5-941a-6ec624dea5ba admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.642583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247dab-0716-43b5-941a-6ec624dea5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.642857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11247dab-0716-43b5-941a-6ec624dea5ba admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.643471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11247dab-0716-43b5-941a-6ec624dea5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:00.645415 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11247dab-0716-43b5-941a-6ec624dea5ba admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:00.666188 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1390/2747] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:02:00 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:00.670322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56605351-9fa6-4766-9be5-ca15e5b0b66a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:00.680500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56605351-9fa6-4766-9be5-ca15e5b0b66a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:00.684998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620a4d-6af3-4168-8c32-7fa0e9921da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:00.685618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620a4d-6af3-4168-8c32-7fa0e9921da9 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.685895 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620a4d-6af3-4168-8c32-7fa0e9921da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.686163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620a4d-6af3-4168-8c32-7fa0e9921da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.686798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50620a4d-6af3-4168-8c32-7fa0e9921da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:00.688403 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50620a4d-6af3-4168-8c32-7fa0e9921da9 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:00.698845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1358/2748] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:02:00.702549 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11247dab-0716-43b5-941a-6ec624dea5ba admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:00.712603 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11247dab-0716-43b5-941a-6ec624dea5ba admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:00.717248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887ca66-2922-472a-a67f-bed48e41822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:00.717825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ca66-2922-472a-a67f-bed48e41822e admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.718102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ca66-2922-472a-a67f-bed48e41822e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.718372 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887ca66-2922-472a-a67f-bed48e41822e admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.719001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887ca66-2922-472a-a67f-bed48e41822e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:00.720548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887ca66-2922-472a-a67f-bed48e41822e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:00.731292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1391/2749] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:02:00.735250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620a4d-6af3-4168-8c32-7fa0e9921da9 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:00.745828 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620a4d-6af3-4168-8c32-7fa0e9921da9 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:00.750343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738840e-5efe-40b9-83c5-fd9e9df00f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:00.751019 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738840e-5efe-40b9-83c5-fd9e9df00f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.751272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738840e-5efe-40b9-83c5-fd9e9df00f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.751540 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738840e-5efe-40b9-83c5-fd9e9df00f24 admin admin] PATH_INFO: `/v3/projects/357a5297c91d49f490a633e626fd209f/users/66f1ba1a19044633bfa4444921f8af38/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.766976 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738840e-5efe-40b9-83c5-fd9e9df00f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=357a5297c91d49f490a633e626fd209f, user_id=66f1ba1a19044633bfa4444921f8af38, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:00.769703 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738840e-5efe-40b9-83c5-fd9e9df00f24 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:00.782491 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1359/2750] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:02:00 2022] PUT /identity/v3/projects/357a5297c91d49f490a633e626fd209f/users/66f1ba1a19044633bfa4444921f8af38/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:00.793158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0654ef-c975-4eb9-b42f-395cbbbd9ebc None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:00.793449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0654ef-c975-4eb9-b42f-395cbbbd9ebc None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:00.793725 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0654ef-c975-4eb9-b42f-395cbbbd9ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:00.821940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4d0654ef-c975-4eb9-b42f-395cbbbd9ebc None None] MFA Rules not processed for user `66f1ba1a19044633bfa4444921f8af38`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:00.823138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4d0654ef-c975-4eb9-b42f-395cbbbd9ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:00.884910 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1392/2751] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:00.889364 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738840e-5efe-40b9-83c5-fd9e9df00f24 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:01.072814 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738840e-5efe-40b9-83c5-fd9e9df00f24 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:01.077398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59295af-b1bb-4421-ab08-5e40d6c66070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:01.077989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59295af-b1bb-4421-ab08-5e40d6c66070 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:01.078264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59295af-b1bb-4421-ab08-5e40d6c66070 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:01.078530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59295af-b1bb-4421-ab08-5e40d6c66070 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:01.079133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59295af-b1bb-4421-ab08-5e40d6c66070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:01.081003 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59295af-b1bb-4421-ab08-5e40d6c66070 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:01.101644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:01.109777 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1360/2752] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:02:00 2022] POST /identity/v3/projects => generated 375 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:01.114186 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0654ef-c975-4eb9-b42f-395cbbbd9ebc None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:01.125005 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0654ef-c975-4eb9-b42f-395cbbbd9ebc None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:01.179310 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54781e6c-c51a-4896-8e64-b71ae4aba683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:01.179899 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54781e6c-c51a-4896-8e64-b71ae4aba683 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:01.180171 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54781e6c-c51a-4896-8e64-b71ae4aba683 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:01.180439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54781e6c-c51a-4896-8e64-b71ae4aba683 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:01.181082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54781e6c-c51a-4896-8e64-b71ae4aba683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:01.182970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54781e6c-c51a-4896-8e64-b71ae4aba683 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:01.203297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1393/2753] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:02:01 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:01.207291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59295af-b1bb-4421-ab08-5e40d6c66070 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:01.218042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59295af-b1bb-4421-ab08-5e40d6c66070 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:01.222722 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804119c-4502-4b50-8152-26b795871099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:01.223309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804119c-4502-4b50-8152-26b795871099 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:01.223585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804119c-4502-4b50-8152-26b795871099 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:01.223877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804119c-4502-4b50-8152-26b795871099 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:01.224501 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804119c-4502-4b50-8152-26b795871099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:01.226684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804119c-4502-4b50-8152-26b795871099 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:01.236739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1361/2754] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:02:01.240901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54781e6c-c51a-4896-8e64-b71ae4aba683 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:01.250597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54781e6c-c51a-4896-8e64-b71ae4aba683 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:01.255045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d649d-bf39-4a6b-8a36-b6f327560c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:01.255675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d649d-bf39-4a6b-8a36-b6f327560c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:01.256428 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d649d-bf39-4a6b-8a36-b6f327560c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:01.256705 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d649d-bf39-4a6b-8a36-b6f327560c62 admin admin] PATH_INFO: `/v3/projects/b78c1a1ae6244dd6b98d8a720a7d6a6e/users/ffa809f2ee864c94b449a3974d345345/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:01.271191 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d649d-bf39-4a6b-8a36-b6f327560c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78c1a1ae6244dd6b98d8a720a7d6a6e, user_id=ffa809f2ee864c94b449a3974d345345, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:01.273981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d649d-bf39-4a6b-8a36-b6f327560c62 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:01.288241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1394/2755] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:02:01 2022] PUT /identity/v3/projects/b78c1a1ae6244dd6b98d8a720a7d6a6e/users/ffa809f2ee864c94b449a3974d345345/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:01.292057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804119c-4502-4b50-8152-26b795871099 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:01.301792 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804119c-4502-4b50-8152-26b795871099 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:01.355279 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45280d3-bdd2-4516-81a4-909e52da5538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:01.355799 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45280d3-bdd2-4516-81a4-909e52da5538 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:01.356073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45280d3-bdd2-4516-81a4-909e52da5538 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:01.356338 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45280d3-bdd2-4516-81a4-909e52da5538 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:01.356954 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45280d3-bdd2-4516-81a4-909e52da5538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:01.358536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45280d3-bdd2-4516-81a4-909e52da5538 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:01.368582 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1362/2756] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 02:02:01.372636 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d649d-bf39-4a6b-8a36-b6f327560c62 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:01.383274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d649d-bf39-4a6b-8a36-b6f327560c62 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:01.389988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:01.390632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:01.390909 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:01.391178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] PATH_INFO: `/v3/projects/b78c1a1ae6244dd6b98d8a720a7d6a6e/users/ffa809f2ee864c94b449a3974d345345/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:01.393099 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78c1a1ae6244dd6b98d8a720a7d6a6e, user_id=ffa809f2ee864c94b449a3974d345345, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:01.395709 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:01.405904 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1395/2757] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:02:01 2022] PUT /identity/v3/projects/b78c1a1ae6244dd6b98d8a720a7d6a6e/users/ffa809f2ee864c94b449a3974d345345/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:01.417545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef27bb-be14-42b3-92d1-d8ad8ecf2013 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:01.417834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef27bb-be14-42b3-92d1-d8ad8ecf2013 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:01.418107 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef27bb-be14-42b3-92d1-d8ad8ecf2013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:01.445860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b7ef27bb-be14-42b3-92d1-d8ad8ecf2013 None None] MFA Rules not processed for user `ffa809f2ee864c94b449a3974d345345`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:01.447050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b7ef27bb-be14-42b3-92d1-d8ad8ecf2013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:01.512460 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1363/2758] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 02:02:01.523842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:01.525505 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:01.538254 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:01.584907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de141c3e-ba0a-4830-b4e3-6fc14dc1c58b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:01.588553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb0727-7bd8-41c3-8ef5-709196884de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:01.589189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb0727-7bd8-41c3-8ef5-709196884de9 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:01.589494 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb0727-7bd8-41c3-8ef5-709196884de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:01.589761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb0727-7bd8-41c3-8ef5-709196884de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:01.644558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb0727-7bd8-41c3-8ef5-709196884de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:01.645911 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb0727-7bd8-41c3-8ef5-709196884de9 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:01.654448 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1396/2759] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:02:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:02:04.188398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef27bb-be14-42b3-92d1-d8ad8ecf2013 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:04.250974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef27bb-be14-42b3-92d1-d8ad8ecf2013 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:04.255432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:04.256559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:04.256832 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:04.257122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] PATH_INFO: `/v3/users/66f1ba1a19044633bfa4444921f8af38` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:04.258235 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f1ba1a19044633bfa4444921f8af38)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:04.260201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:04.270223 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=100 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:04.306427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f1ba1a19044633bfa4444921f8af38'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.307105 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f1ba1a19044633bfa4444921f8af38'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.314526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f1ba1a19044633bfa4444921f8af38'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.326900 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f1ba1a19044633bfa4444921f8af38'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.333612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f1ba1a19044633bfa4444921f8af38'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.339948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1364/2760] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:02:04 2022] DELETE /identity/v3/users/66f1ba1a19044633bfa4444921f8af38 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:04.344413 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad81e-a3ad-4f71-9eee-8f2ecee2a1a5 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:04.344913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad81e-a3ad-4f71-9eee-8f2ecee2a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:04.345018 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad81e-a3ad-4f71-9eee-8f2ecee2a1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:04.364776 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-8d3ad81e-a3ad-4f71-9eee-8f2ecee2a1a5 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:04.366130 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8d3ad81e-a3ad-4f71-9eee-8f2ecee2a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:04.423140 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1397/2761] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:04.433005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:04.434425 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:04.494768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362587d-0a5d-4c2e-b9b2-475f542da60a admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:04.499343 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb60020-ef69-4250-b3de-87a0b977fcde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:04.500083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb60020-ef69-4250-b3de-87a0b977fcde service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:04.502489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb60020-ef69-4250-b3de-87a0b977fcde service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:04.502489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb60020-ef69-4250-b3de-87a0b977fcde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:04.512965 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb60020-ef69-4250-b3de-87a0b977fcde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:04.515242 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb60020-ef69-4250-b3de-87a0b977fcde service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:04.529732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1365/2762] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 02:02:04.788499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad81e-a3ad-4f71-9eee-8f2ecee2a1a5 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:04.790034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8d3ad81e-a3ad-4f71-9eee-8f2ecee2a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:04.803233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad81e-a3ad-4f71-9eee-8f2ecee2a1a5 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:04.808565 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:04.809833 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:04.810291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:04.810569 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] PATH_INFO: `/v3/projects/357a5297c91d49f490a633e626fd209f` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:04.811692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] RBAC: Authorizing `identity:delete_project(project_id=357a5297c91d49f490a633e626fd209f)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:04.813671 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:04.868265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '357a5297c91d49f490a633e626fd209f', 'request_id': 'req-d6a2ef04-052d-4007-9dc9-c1f5d157762b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.875770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '357a5297c91d49f490a633e626fd209f', 'request_id': 'req-d6a2ef04-052d-4007-9dc9-c1f5d157762b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.877812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:04.883001 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '357a5297c91d49f490a633e626fd209f', 'request_id': 'req-d6a2ef04-052d-4007-9dc9-c1f5d157762b'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.895663 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] The token cache is being invalidate because project 357a5297c91d49f490a633e626fd209f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:02:04.895948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.896628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:04.898196 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1398/2763] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:02:04 2022] DELETE /identity/v3/projects/357a5297c91d49f490a633e626fd209f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:04.903388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb60020-ef69-4250-b3de-87a0b977fcde service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:04.912344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb60020-ef69-4250-b3de-87a0b977fcde service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:04.967116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:04.967692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:04.967963 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:04.968228 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] PATH_INFO: `/v3/users/ffa809f2ee864c94b449a3974d345345` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:04.969355 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa809f2ee864c94b449a3974d345345)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:04.971256 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:05.004613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:05.018373 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa809f2ee864c94b449a3974d345345'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.019021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa809f2ee864c94b449a3974d345345'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.026054 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa809f2ee864c94b449a3974d345345'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.037939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa809f2ee864c94b449a3974d345345'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.044507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ac60d3c2-8c1b-43bf-ba02-824a64f3bf5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa809f2ee864c94b449a3974d345345'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.050187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1366/2764] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:02:04 2022] DELETE /identity/v3/users/ffa809f2ee864c94b449a3974d345345 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:02:05.325566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:05.327020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:05.340809 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2ef04-052d-4007-9dc9-c1f5d157762b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:05.397093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:05.397720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.398000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.398267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] PATH_INFO: `/v3/projects/b78c1a1ae6244dd6b98d8a720a7d6a6e` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.399386 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b78c1a1ae6244dd6b98d8a720a7d6a6e)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:05.401279 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:05.449608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b78c1a1ae6244dd6b98d8a720a7d6a6e', 'request_id': 'req-8550561c-db2d-4639-9d82-568cf1e6e687'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.456112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b78c1a1ae6244dd6b98d8a720a7d6a6e', 'request_id': 'req-8550561c-db2d-4639-9d82-568cf1e6e687'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.462098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b78c1a1ae6244dd6b98d8a720a7d6a6e', 'request_id': 'req-8550561c-db2d-4639-9d82-568cf1e6e687'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.474728 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] The token cache is being invalidate because project b78c1a1ae6244dd6b98d8a720a7d6a6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:02:05.475014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.475626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-8550561c-db2d-4639-9d82-568cf1e6e687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:05.477080 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1399/2765] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:02:05 2022] DELETE /identity/v3/projects/b78c1a1ae6244dd6b98d8a720a7d6a6e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:05.482063 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d91c8-e7f8-4224-8907-027627a29944 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.482555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d91c8-e7f8-4224-8907-027627a29944 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.482657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d91c8-e7f8-4224-8907-027627a29944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.503670 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e22d91c8-e7f8-4224-8907-027627a29944 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:05.504778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e22d91c8-e7f8-4224-8907-027627a29944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:05.561544 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1367/2766] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:05.570153 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0b7b4-4050-4516-9191-afd6ef83dad7 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.570431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0b7b4-4050-4516-9191-afd6ef83dad7 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.571113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0b7b4-4050-4516-9191-afd6ef83dad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.590138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-8bf0b7b4-4050-4516-9191-afd6ef83dad7 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:05.591306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-8bf0b7b4-4050-4516-9191-afd6ef83dad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:05.599244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1400/2767] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 02:02:05.603363 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d91c8-e7f8-4224-8907-027627a29944 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:05.612883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d91c8-e7f8-4224-8907-027627a29944 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:05.618233 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:05.618819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.619093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.619359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.620026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:02:05.620328 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:05.621916 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:05.633717 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1368/2768] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:02:05.637760 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0b7b4-4050-4516-9191-afd6ef83dad7 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:05.646813 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0b7b4-4050-4516-9191-afd6ef83dad7 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:05.651394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f2cfa-b9e2-4a1e-abac-b32ee3359826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:05.651969 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2cfa-b9e2-4a1e-abac-b32ee3359826 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.652238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2cfa-b9e2-4a1e-abac-b32ee3359826 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.652504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f2cfa-b9e2-4a1e-abac-b32ee3359826 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.653157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f2cfa-b9e2-4a1e-abac-b32ee3359826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:05.655011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f2cfa-b9e2-4a1e-abac-b32ee3359826 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:05.681647 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1401/2769] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:02:05 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 02:02:05.685637 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:05.695226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e4c5f-bc91-4abb-bc01-b64f90cbc4cc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:05.749682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1931b9b-0501-415f-aa6b-bedae45fd72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:05.750267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1931b9b-0501-415f-aa6b-bedae45fd72b admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.750542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1931b9b-0501-415f-aa6b-bedae45fd72b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.750809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1931b9b-0501-415f-aa6b-bedae45fd72b admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.751413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1931b9b-0501-415f-aa6b-bedae45fd72b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:05.753301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1931b9b-0501-415f-aa6b-bedae45fd72b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:05.774530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1369/2770] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:02:05 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:05.778495 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f2cfa-b9e2-4a1e-abac-b32ee3359826 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:05.788644 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f2cfa-b9e2-4a1e-abac-b32ee3359826 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:05.793210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf8d86-4220-4531-b6f7-697f9209f390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:05.793792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf8d86-4220-4531-b6f7-697f9209f390 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.794113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf8d86-4220-4531-b6f7-697f9209f390 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.794376 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf8d86-4220-4531-b6f7-697f9209f390 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.795027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf8d86-4220-4531-b6f7-697f9209f390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:05.796581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf8d86-4220-4531-b6f7-697f9209f390 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:05.806645 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1402/2771] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 02:02:05.810312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1931b9b-0501-415f-aa6b-bedae45fd72b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:05.820457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1931b9b-0501-415f-aa6b-bedae45fd72b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:05.825026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153ee23-7621-4cdd-a6c5-2b6808694105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:05.825726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153ee23-7621-4cdd-a6c5-2b6808694105 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.826001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153ee23-7621-4cdd-a6c5-2b6808694105 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.826272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153ee23-7621-4cdd-a6c5-2b6808694105 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.826897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153ee23-7621-4cdd-a6c5-2b6808694105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:05.828437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153ee23-7621-4cdd-a6c5-2b6808694105 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:05.838876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1370/2772] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:02:05.842888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf8d86-4220-4531-b6f7-697f9209f390 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:05.853104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf8d86-4220-4531-b6f7-697f9209f390 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:05.857606 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:05.859239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.859514 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.859785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] PATH_INFO: `/v3/projects/34ed79839b484bbeab2833514035c174/users/faf50c42d46f41d09489f6f81753cdbc/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.875152 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ed79839b484bbeab2833514035c174, user_id=faf50c42d46f41d09489f6f81753cdbc, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:05.877792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:05.890778 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1403/2773] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:02:05 2022] PUT /identity/v3/projects/34ed79839b484bbeab2833514035c174/users/faf50c42d46f41d09489f6f81753cdbc/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:05.900901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd111f-6945-45a9-a927-ff14c38dbd58 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:05.901236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd111f-6945-45a9-a927-ff14c38dbd58 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:05.901506 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd111f-6945-45a9-a927-ff14c38dbd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:05.930562 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-e7cd111f-6945-45a9-a927-ff14c38dbd58 None None] MFA Rules not processed for user `faf50c42d46f41d09489f6f81753cdbc`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:05.931746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-e7cd111f-6945-45a9-a927-ff14c38dbd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:05.935623 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:05.995936 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1371/2774] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:06.006362 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:06.008002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:06.021525 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:06.065652 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f66ba-ada1-49e4-b32e-b33bed2de723 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:06.069639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6002b-adff-42fb-92ee-16df1f7a3d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:06.070236 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6002b-adff-42fb-92ee-16df1f7a3d5a service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:06.070511 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6002b-adff-42fb-92ee-16df1f7a3d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:06.070781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6002b-adff-42fb-92ee-16df1f7a3d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:06.078811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6002b-adff-42fb-92ee-16df1f7a3d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:06.080202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6002b-adff-42fb-92ee-16df1f7a3d5a service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:06.089364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1404/2775] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:02:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:02:09.100918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd111f-6945-45a9-a927-ff14c38dbd58 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:09.161173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd111f-6945-45a9-a927-ff14c38dbd58 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:09.166128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:09.166743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:09.167020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:09.167291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] PATH_INFO: `/v3/users/faf50c42d46f41d09489f6f81753cdbc` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:09.168389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faf50c42d46f41d09489f6f81753cdbc)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:09.170315 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:09.215492 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faf50c42d46f41d09489f6f81753cdbc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.217201 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faf50c42d46f41d09489f6f81753cdbc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.225651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faf50c42d46f41d09489f6f81753cdbc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.238075 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faf50c42d46f41d09489f6f81753cdbc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.247580 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faf50c42d46f41d09489f6f81753cdbc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.253606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1372/2776] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:02:09 2022] DELETE /identity/v3/users/faf50c42d46f41d09489f6f81753cdbc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:09.258268 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816023fa-4b5f-48b7-8bd0-eaa9b6522700 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:09.258930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816023fa-4b5f-48b7-8bd0-eaa9b6522700 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:09.258930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816023fa-4b5f-48b7-8bd0-eaa9b6522700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:09.277927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-816023fa-4b5f-48b7-8bd0-eaa9b6522700 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:09.279102 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-816023fa-4b5f-48b7-8bd0-eaa9b6522700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:09.336398 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1405/2777] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 02:02:09.346028 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:09.347545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:09.375784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:09.408518 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56421b14-1668-4ff2-b997-9d76792258b0 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:09.413181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d94a53-0db1-49e5-932c-6aaa24ec173a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:09.413726 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d94a53-0db1-49e5-932c-6aaa24ec173a service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:09.414005 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d94a53-0db1-49e5-932c-6aaa24ec173a service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:09.414271 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d94a53-0db1-49e5-932c-6aaa24ec173a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:09.422983 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d94a53-0db1-49e5-932c-6aaa24ec173a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:09.424388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d94a53-0db1-49e5-932c-6aaa24ec173a service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:09.433659 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1373/2778] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:02:09.700577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816023fa-4b5f-48b7-8bd0-eaa9b6522700 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:09.702069 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-816023fa-4b5f-48b7-8bd0-eaa9b6522700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:09.717723 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816023fa-4b5f-48b7-8bd0-eaa9b6522700 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:09.722479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:09.723107 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:09.723365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:09.723732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] PATH_INFO: `/v3/projects/34ed79839b484bbeab2833514035c174` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:09.724849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ed79839b484bbeab2833514035c174)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:09.726730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:09.772230 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ed79839b484bbeab2833514035c174', 'request_id': 'req-0992e666-cda4-4fc1-9714-783536f1d5d8'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.778154 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ed79839b484bbeab2833514035c174', 'request_id': 'req-0992e666-cda4-4fc1-9714-783536f1d5d8'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.783626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ed79839b484bbeab2833514035c174', 'request_id': 'req-0992e666-cda4-4fc1-9714-783536f1d5d8'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.796203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] The token cache is being invalidate because project 34ed79839b484bbeab2833514035c174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:02:09.796491 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.797138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-0992e666-cda4-4fc1-9714-783536f1d5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:09.798589 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1406/2779] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:02:09 2022] DELETE /identity/v3/projects/34ed79839b484bbeab2833514035c174 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:09.804185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbb87c-1388-49a3-aece-64239050cb57 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:09.804463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbb87c-1388-49a3-aece-64239050cb57 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:09.804730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbb87c-1388-49a3-aece-64239050cb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:09.823643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-8afbb87c-1388-49a3-aece-64239050cb57 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:09.824815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-8afbb87c-1388-49a3-aece-64239050cb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:09.881216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1374/2780] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:09.943162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dee88-a9ae-45ed-9620-bfedb81f6cdc None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:09.943390 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dee88-a9ae-45ed-9620-bfedb81f6cdc None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:09.943611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53dee88-a9ae-45ed-9620-bfedb81f6cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:09.964468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-d53dee88-a9ae-45ed-9620-bfedb81f6cdc None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:09.965690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-d53dee88-a9ae-45ed-9620-bfedb81f6cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:09.973587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1407/2781] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:09.977812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbb87c-1388-49a3-aece-64239050cb57 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:09.987374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbb87c-1388-49a3-aece-64239050cb57 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:09.992710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:09.993336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:09.993619 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:09.993891 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:09.994534 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:02:09.995042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:09.996629 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:10.008567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1375/2782] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:02:10.012770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dee88-a9ae-45ed-9620-bfedb81f6cdc None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:10.021793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53dee88-a9ae-45ed-9620-bfedb81f6cdc None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:10.026376 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1557e6-7c85-456d-99fb-d5f3d785c9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:10.026966 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1557e6-7c85-456d-99fb-d5f3d785c9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:10.027237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1557e6-7c85-456d-99fb-d5f3d785c9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:10.027504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1557e6-7c85-456d-99fb-d5f3d785c9c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:10.028114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1557e6-7c85-456d-99fb-d5f3d785c9c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:10.030020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1557e6-7c85-456d-99fb-d5f3d785c9c6 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:10.056294 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1408/2783] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:02:10 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:10.060386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:10.070945 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2166f-c135-40a9-a13e-013adf7e939c admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:10.126334 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727993dc-8726-406f-b56a-21e9b2a5eb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:10.126939 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727993dc-8726-406f-b56a-21e9b2a5eb1b admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:10.127184 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727993dc-8726-406f-b56a-21e9b2a5eb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:10.127451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727993dc-8726-406f-b56a-21e9b2a5eb1b admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:10.128131 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727993dc-8726-406f-b56a-21e9b2a5eb1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:10.130052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727993dc-8726-406f-b56a-21e9b2a5eb1b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:10.150982 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1376/2784] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:02:10 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:10.155243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1557e6-7c85-456d-99fb-d5f3d785c9c6 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:10.165283 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1557e6-7c85-456d-99fb-d5f3d785c9c6 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:10.169774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922617d6-b1dc-4249-b882-49b905e112d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:10.170357 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922617d6-b1dc-4249-b882-49b905e112d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:10.170626 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922617d6-b1dc-4249-b882-49b905e112d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:10.170915 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922617d6-b1dc-4249-b882-49b905e112d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:10.171513 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922617d6-b1dc-4249-b882-49b905e112d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:10.173084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922617d6-b1dc-4249-b882-49b905e112d9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:10.175123 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:10.184988 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1409/2785] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:02:10.188339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727993dc-8726-406f-b56a-21e9b2a5eb1b admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:10.199145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727993dc-8726-406f-b56a-21e9b2a5eb1b admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:10.203661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e856828-29dc-452b-8a6b-557ec1de0c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:10.204374 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856828-29dc-452b-8a6b-557ec1de0c32 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:10.204651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856828-29dc-452b-8a6b-557ec1de0c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:10.204920 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856828-29dc-452b-8a6b-557ec1de0c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:10.205596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856828-29dc-452b-8a6b-557ec1de0c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:10.207167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856828-29dc-452b-8a6b-557ec1de0c32 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:10.218684 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1377/2786] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:02:10.222814 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922617d6-b1dc-4249-b882-49b905e112d9 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:10.233047 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922617d6-b1dc-4249-b882-49b905e112d9 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:10.237539 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:10.238443 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:10.238703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:10.238967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] PATH_INFO: `/v3/projects/d1b0b7e09fb34a5d84d1ae9eb14b76fa/users/a4a347065d0a48c2b51fddb5be22320e/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:10.253479 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b0b7e09fb34a5d84d1ae9eb14b76fa, user_id=a4a347065d0a48c2b51fddb5be22320e, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:10.256110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:10.268912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1410/2787] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:02:10 2022] PUT /identity/v3/projects/d1b0b7e09fb34a5d84d1ae9eb14b76fa/users/a4a347065d0a48c2b51fddb5be22320e/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:10.279112 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07777807-7874-4830-ac29-2ede7085b8aa None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:10.279397 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07777807-7874-4830-ac29-2ede7085b8aa None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:10.279668 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07777807-7874-4830-ac29-2ede7085b8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:10.307205 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-07777807-7874-4830-ac29-2ede7085b8aa None None] MFA Rules not processed for user `a4a347065d0a48c2b51fddb5be22320e`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:10.308685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-07777807-7874-4830-ac29-2ede7085b8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:10.328894 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:10.372434 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1378/2788] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:10.384304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:10.386831 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:10.443914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c6288-19c5-480e-8d4f-02743741a3e1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:10.447877 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c31d0-3180-4d38-a5cc-81732128464b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:10.448553 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c31d0-3180-4d38-a5cc-81732128464b service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:10.448846 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c31d0-3180-4d38-a5cc-81732128464b service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:10.449265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c31d0-3180-4d38-a5cc-81732128464b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:10.458577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c31d0-3180-4d38-a5cc-81732128464b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:10.460587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c31d0-3180-4d38-a5cc-81732128464b service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:10.469301 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1411/2789] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:02:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:02:13.014451 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07777807-7874-4830-ac29-2ede7085b8aa None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:13.074641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07777807-7874-4830-ac29-2ede7085b8aa None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:13.079270 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:13.079889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.080165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.080438 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] PATH_INFO: `/v3/users/a4a347065d0a48c2b51fddb5be22320e` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.081559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a347065d0a48c2b51fddb5be22320e)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:13.083449 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:13.128175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a347065d0a48c2b51fddb5be22320e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.128819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a347065d0a48c2b51fddb5be22320e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.136561 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a347065d0a48c2b51fddb5be22320e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.148712 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a347065d0a48c2b51fddb5be22320e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.155587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a347065d0a48c2b51fddb5be22320e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.161525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1379/2790] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:02:13 2022] DELETE /identity/v3/users/a4a347065d0a48c2b51fddb5be22320e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:13.166593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1d110-4b9b-409a-8ebb-6a19ba9f50ed None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.167071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1d110-4b9b-409a-8ebb-6a19ba9f50ed None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.167187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1d110-4b9b-409a-8ebb-6a19ba9f50ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.186028 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-f7d1d110-4b9b-409a-8ebb-6a19ba9f50ed None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:13.187213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f7d1d110-4b9b-409a-8ebb-6a19ba9f50ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:13.246679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1412/2791] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:13.256855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:13.258274 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:13.303140 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:13.316124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699ee39-d20a-40cb-ba94-476b7ba3a0d7 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:13.320237 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8b70d-973e-4b7f-88c4-5852e4e377c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:13.320859 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8b70d-973e-4b7f-88c4-5852e4e377c7 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.321166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8b70d-973e-4b7f-88c4-5852e4e377c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.321439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8b70d-973e-4b7f-88c4-5852e4e377c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.330352 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8b70d-973e-4b7f-88c4-5852e4e377c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:13.331750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8b70d-973e-4b7f-88c4-5852e4e377c7 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:13.341585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1380/2792] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:02:13.601992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1d110-4b9b-409a-8ebb-6a19ba9f50ed None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:13.603471 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f7d1d110-4b9b-409a-8ebb-6a19ba9f50ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:13.616903 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1d110-4b9b-409a-8ebb-6a19ba9f50ed None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:13.621614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:13.622321 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.622599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.622875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] PATH_INFO: `/v3/projects/d1b0b7e09fb34a5d84d1ae9eb14b76fa` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.624006 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1b0b7e09fb34a5d84d1ae9eb14b76fa)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:13.625929 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:13.672593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1b0b7e09fb34a5d84d1ae9eb14b76fa', 'request_id': 'req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.678661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1b0b7e09fb34a5d84d1ae9eb14b76fa', 'request_id': 'req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.684192 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1b0b7e09fb34a5d84d1ae9eb14b76fa', 'request_id': 'req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.692812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:13.697303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] The token cache is being invalidate because project d1b0b7e09fb34a5d84d1ae9eb14b76fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:02:13.697919 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.699021 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-c62a3f77-43ce-4ac8-9c6b-ae26625d51d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:13.700582 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1413/2793] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:02:13 2022] DELETE /identity/v3/projects/d1b0b7e09fb34a5d84d1ae9eb14b76fa => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:13.705606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8d9ec-a409-4f66-b33e-5480366b27f3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.705889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8d9ec-a409-4f66-b33e-5480366b27f3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.706158 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8d9ec-a409-4f66-b33e-5480366b27f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.726138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-b1f8d9ec-a409-4f66-b33e-5480366b27f3 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:13.727310 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-b1f8d9ec-a409-4f66-b33e-5480366b27f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:13.784409 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1381/2794] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:13.793973 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f97400-e1ef-4de2-9749-6e98a83c8b3e None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.794257 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f97400-e1ef-4de2-9749-6e98a83c8b3e None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.794530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f97400-e1ef-4de2-9749-6e98a83c8b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.814011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-73f97400-e1ef-4de2-9749-6e98a83c8b3e None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:13.815213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-73f97400-e1ef-4de2-9749-6e98a83c8b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:13.823238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1414/2795] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:13.827543 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8d9ec-a409-4f66-b33e-5480366b27f3 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:13.837633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8d9ec-a409-4f66-b33e-5480366b27f3 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:13.842213 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:13.842803 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.843080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.843349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.843981 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:02:13.844280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:13.845873 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:13.858025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1382/2796] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:02:13.862233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f97400-e1ef-4de2-9749-6e98a83c8b3e None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:13.871469 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f97400-e1ef-4de2-9749-6e98a83c8b3e None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:13.876238 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c357b6f-4ec9-4326-96d7-90f88a5cd907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:13.876882 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c357b6f-4ec9-4326-96d7-90f88a5cd907 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.877189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c357b6f-4ec9-4326-96d7-90f88a5cd907 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.877460 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c357b6f-4ec9-4326-96d7-90f88a5cd907 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.878089 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c357b6f-4ec9-4326-96d7-90f88a5cd907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:13.880012 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c357b6f-4ec9-4326-96d7-90f88a5cd907 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:13.907580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1415/2797] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:02:13 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:13.912590 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:13.922908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c576ed2-ab9e-49cb-993f-00a123b92835 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:13.977227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62022fe8-43f6-43fd-becd-6b78a9dc6d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:13.977762 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62022fe8-43f6-43fd-becd-6b78a9dc6d3f admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:13.978053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62022fe8-43f6-43fd-becd-6b78a9dc6d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:13.978312 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62022fe8-43f6-43fd-becd-6b78a9dc6d3f admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:13.978922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62022fe8-43f6-43fd-becd-6b78a9dc6d3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:13.980835 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62022fe8-43f6-43fd-becd-6b78a9dc6d3f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:14.006601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1383/2798] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:02:13 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:02:14.010743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c357b6f-4ec9-4326-96d7-90f88a5cd907 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:14.021998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c357b6f-4ec9-4326-96d7-90f88a5cd907 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:14.026563 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b35bbc-a15d-4775-a49d-ab5a25ceea85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:14.027146 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b35bbc-a15d-4775-a49d-ab5a25ceea85 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:14.027421 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b35bbc-a15d-4775-a49d-ab5a25ceea85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:14.027690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b35bbc-a15d-4775-a49d-ab5a25ceea85 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:14.028318 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b35bbc-a15d-4775-a49d-ab5a25ceea85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:14.029922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b35bbc-a15d-4775-a49d-ab5a25ceea85 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:14.039874 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1416/2799] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:02:14.043794 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62022fe8-43f6-43fd-becd-6b78a9dc6d3f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:14.053567 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62022fe8-43f6-43fd-becd-6b78a9dc6d3f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:14.058082 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b8f55-f24a-44dc-aefd-dc896d80a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:14.058720 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8f55-f24a-44dc-aefd-dc896d80a553 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:14.058999 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8f55-f24a-44dc-aefd-dc896d80a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:14.059267 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b8f55-f24a-44dc-aefd-dc896d80a553 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:14.059896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b8f55-f24a-44dc-aefd-dc896d80a553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:14.061499 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b8f55-f24a-44dc-aefd-dc896d80a553 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:14.072329 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1384/2800] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:02:14.076199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b35bbc-a15d-4775-a49d-ab5a25ceea85 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:14.086472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b35bbc-a15d-4775-a49d-ab5a25ceea85 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:14.091044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:14.091667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:14.091939 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:14.092206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] PATH_INFO: `/v3/projects/9ecd46e918d4492493a2fb8380a12cb9/users/b574277e7d7143f1a14f2384e113a0f3/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:14.106147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ecd46e918d4492493a2fb8380a12cb9, user_id=b574277e7d7143f1a14f2384e113a0f3, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:14.108720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:14.121455 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1417/2801] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:02:14 2022] PUT /identity/v3/projects/9ecd46e918d4492493a2fb8380a12cb9/users/b574277e7d7143f1a14f2384e113a0f3/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:02:14.131996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41216352-ac56-4a95-a209-128c98c10938 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:14.132283 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41216352-ac56-4a95-a209-128c98c10938 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:14.132552 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41216352-ac56-4a95-a209-128c98c10938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:14.160076 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-41216352-ac56-4a95-a209-128c98c10938 None None] MFA Rules not processed for user `b574277e7d7143f1a14f2384e113a0f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:14.161325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-41216352-ac56-4a95-a209-128c98c10938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:14.199586 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:14.225842 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1385/2802] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:14 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:14.238848 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:14.240658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:14.297462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df7637a-7fe1-4e86-ad42-1392ad03a566 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:14.301084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43908eb-7cbf-4369-bd33-a2802214980f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:14.301697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43908eb-7cbf-4369-bd33-a2802214980f service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:14.301976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43908eb-7cbf-4369-bd33-a2802214980f service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:14.302291 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43908eb-7cbf-4369-bd33-a2802214980f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:14.310397 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43908eb-7cbf-4369-bd33-a2802214980f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:14.311844 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43908eb-7cbf-4369-bd33-a2802214980f service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:14.321045 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1418/2803] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:02:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:02:22.386537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41216352-ac56-4a95-a209-128c98c10938 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:22.449126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41216352-ac56-4a95-a209-128c98c10938 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:22.453530 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:22.454143 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:22.454413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:22.454677 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] PATH_INFO: `/v3/users/b574277e7d7143f1a14f2384e113a0f3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:22.455770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b574277e7d7143f1a14f2384e113a0f3)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:22.457679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:22.502380 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b574277e7d7143f1a14f2384e113a0f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:22.503024 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b574277e7d7143f1a14f2384e113a0f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:22.511009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b574277e7d7143f1a14f2384e113a0f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:22.523474 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b574277e7d7143f1a14f2384e113a0f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:22.530187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b574277e7d7143f1a14f2384e113a0f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:22.535889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1386/2804] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:02:22 2022] DELETE /identity/v3/users/b574277e7d7143f1a14f2384e113a0f3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:22.540323 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a88d4-91bc-4cc3-b66b-ff51f9df0f8a None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:22.540781 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a88d4-91bc-4cc3-b66b-ff51f9df0f8a None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:22.540884 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a88d4-91bc-4cc3-b66b-ff51f9df0f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:22.559341 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-964a88d4-91bc-4cc3-b66b-ff51f9df0f8a None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:02:22.560517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-964a88d4-91bc-4cc3-b66b-ff51f9df0f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:22.618832 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1419/2805] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:02:22.628225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:22.629739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:22.685616 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba583732-5cf4-456e-89fc-5f729c15c104 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:22.689276 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff7b54-4a07-401a-bb79-63685f8b31e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:22.689948 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff7b54-4a07-401a-bb79-63685f8b31e1 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:22.690234 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff7b54-4a07-401a-bb79-63685f8b31e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:22.690505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff7b54-4a07-401a-bb79-63685f8b31e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:22.701230 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff7b54-4a07-401a-bb79-63685f8b31e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:22.702652 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff7b54-4a07-401a-bb79-63685f8b31e1 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:22.712068 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1387/2806] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:02:22.973459 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a88d4-91bc-4cc3-b66b-ff51f9df0f8a None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:02:22.974962 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-964a88d4-91bc-4cc3-b66b-ff51f9df0f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:02:22.987290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a88d4-91bc-4cc3-b66b-ff51f9df0f8a None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:02:22.992178 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:02:22.992810 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:02:22.993112 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:02:22.993384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] PATH_INFO: `/v3/projects/9ecd46e918d4492493a2fb8380a12cb9` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:02:22.994694 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ecd46e918d4492493a2fb8380a12cb9)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:02:22.996555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:02:23.034769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:23.046116 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ecd46e918d4492493a2fb8380a12cb9', 'request_id': 'req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:23.052113 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ecd46e918d4492493a2fb8380a12cb9', 'request_id': 'req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:23.058206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ecd46e918d4492493a2fb8380a12cb9', 'request_id': 'req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:23.071239 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] The token cache is being invalidate because project 9ecd46e918d4492493a2fb8380a12cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:02:23.071521 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:23.072174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:02:23.073692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1420/2807] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:02:22 2022] DELETE /identity/v3/projects/9ecd46e918d4492493a2fb8380a12cb9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:02:32.707860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:02:33.067340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:03:20.440753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b0c9-b331-4e45-97dc-3e82601de3eb None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:03:20.441309 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b0c9-b331-4e45-97dc-3e82601de3eb None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:03:20.441435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b0c9-b331-4e45-97dc-3e82601de3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:03:20.461177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-d679b0c9-b331-4e45-97dc-3e82601de3eb None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:03:20.462348 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d679b0c9-b331-4e45-97dc-3e82601de3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:03:20.515964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1388/2808] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 02:03:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:03:20.562149 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:03:20.563529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:03:20.635849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db77ca-a9fc-4320-9ab0-a0d622a70fd8 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:03:20.642013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3885c-b510-4791-8fa1-69a915f01a9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:03:20.642614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3885c-b510-4791-8fa1-69a915f01a9d service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:03:20.642895 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3885c-b510-4791-8fa1-69a915f01a9d service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:03:20.643162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3885c-b510-4791-8fa1-69a915f01a9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:03:20.654148 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3885c-b510-4791-8fa1-69a915f01a9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:03:20.655842 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e3885c-b510-4791-8fa1-69a915f01a9d service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:03:20.666992 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1421/2809] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 02:03:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 02:03:26.796035 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b0c9-b331-4e45-97dc-3e82601de3eb None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:03:26.797554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-d679b0c9-b331-4e45-97dc-3e82601de3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:03:26.865774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b0c9-b331-4e45-97dc-3e82601de3eb None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:03:26.870171 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207beee-f7fa-4e42-a8c5-70b306175044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:03:26.870766 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207beee-f7fa-4e42-a8c5-70b306175044 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:03:26.871036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207beee-f7fa-4e42-a8c5-70b306175044 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:03:26.871304 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207beee-f7fa-4e42-a8c5-70b306175044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:03:26.873022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3207beee-f7fa-4e42-a8c5-70b306175044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:03:26.972711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207beee-f7fa-4e42-a8c5-70b306175044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:03:26.974744 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207beee-f7fa-4e42-a8c5-70b306175044 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:03:26.987664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1389/2810] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:03:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:03:28.195807 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3885c-b510-4791-8fa1-69a915f01a9d service swift] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:03:28.209400 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3885c-b510-4791-8fa1-69a915f01a9d service swift] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:03:28.212990 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:03:28.213633 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:03:28.213912 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:03:28.214182 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:03:28.215995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:03:28.233277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:03:28.235026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:03:28.248901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1422/2811] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:03:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:03:35.942536 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350ab11-e0ab-4df7-b915-8ee4950e70f6 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:03:35.943021 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350ab11-e0ab-4df7-b915-8ee4950e70f6 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:03:35.943139 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350ab11-e0ab-4df7-b915-8ee4950e70f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:03:35.962463 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6350ab11-e0ab-4df7-b915-8ee4950e70f6 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:03:35.963645 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6350ab11-e0ab-4df7-b915-8ee4950e70f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:03:35.970734 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1390/2812] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 02:03:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:03:35.982583 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:03:35.993763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383364-12d9-4d25-b16b-d363b070c097 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:03:35.998689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037f359-598e-49a9-aa2e-8988e00d4af8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:03:35.999274 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037f359-598e-49a9-aa2e-8988e00d4af8 service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:03:35.999544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037f359-598e-49a9-aa2e-8988e00d4af8 service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:03:35.999811 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037f359-598e-49a9-aa2e-8988e00d4af8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:03:36.008145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037f359-598e-49a9-aa2e-8988e00d4af8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:03:36.009743 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037f359-598e-49a9-aa2e-8988e00d4af8 service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:03:36.017564 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1423/2813] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 02:03:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 02:03:45.956350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=82 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:03:46.014332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=44 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:03:54.186420 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08958e5e-5c45-4bc8-af85-21d282670d7a None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:03:54.186924 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08958e5e-5c45-4bc8-af85-21d282670d7a None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:03:54.187036 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08958e5e-5c45-4bc8-af85-21d282670d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:03:54.206653 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-08958e5e-5c45-4bc8-af85-21d282670d7a None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:03:54.207865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-08958e5e-5c45-4bc8-af85-21d282670d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:03:54.214955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1391/2814] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 02:03:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:03:54.227250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037f359-598e-49a9-aa2e-8988e00d4af8 service swift] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:03:54.238674 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037f359-598e-49a9-aa2e-8988e00d4af8 service swift] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:03:54.243713 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898547ce-8618-40ac-ba45-a994d95fb2d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:03:54.244298 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898547ce-8618-40ac-ba45-a994d95fb2d7 service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:03:54.244574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898547ce-8618-40ac-ba45-a994d95fb2d7 service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:03:54.244843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898547ce-8618-40ac-ba45-a994d95fb2d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:03:54.253623 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898547ce-8618-40ac-ba45-a994d95fb2d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:03:54.255204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898547ce-8618-40ac-ba45-a994d95fb2d7 service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:03:54.263258 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1424/2815] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 02:03:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 02:04:04.201136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:04:04.259958 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=6 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:04:50.858890 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08958e5e-5c45-4bc8-af85-21d282670d7a None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:04:50.868441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08958e5e-5c45-4bc8-af85-21d282670d7a None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:04:50.872055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:04:50.872644 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:04:50.872918 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:04:50.873225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:04:50.875011 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:04:50.884919 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:04:50.886340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:04:50.895991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1392/2816] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:04:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:04:59.211750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898547ce-8618-40ac-ba45-a994d95fb2d7 service swift] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:04:59.221402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898547ce-8618-40ac-ba45-a994d95fb2d7 service swift] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:04:59.224863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:04:59.225472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:04:59.225747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:04:59.226014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:04:59.227667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:04:59.290795 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:04:59.292195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:04:59.303120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1425/2817] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:00.890573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=6 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:02.334661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:02.337461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:02.395995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:02.397993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:02.414166 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e71d2-58d3-47c2-89ab-214b6d6ff709 service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:02.418921 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:02.419077 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79a01e-1e0e-4e88-9c3b-11826458afff service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:02.419698 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:02.420103 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:02.420384 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] PATH_INFO: `/v3/users/eaaf03c8c05d4f598328ef5c05974e5e` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:02.421519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaaf03c8c05d4f598328ef5c05974e5e)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:02.424007 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:02.424633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:02.425264 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:02.425548 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:02.425816 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:02.428427 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:02.440413 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:02.441947 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:02.452204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1393/2818] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:02.475545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaaf03c8c05d4f598328ef5c05974e5e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.476202 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaaf03c8c05d4f598328ef5c05974e5e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.489300 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaaf03c8c05d4f598328ef5c05974e5e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.496273 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaaf03c8c05d4f598328ef5c05974e5e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.496887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-d4659d03-c54b-428c-9164-bca409107e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaaf03c8c05d4f598328ef5c05974e5e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.509658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1426/2819] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:05:02 2022] DELETE /identity/v3/users/eaaf03c8c05d4f598328ef5c05974e5e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:05:02.686204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:02.699827 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24a912-9f13-4d0b-8f43-329769b0e042 service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:02.758962 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:02.759585 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:02.759863 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:02.760142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] PATH_INFO: `/v3/projects/d2f935cd526946779ecce236c9d47f3f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:02.761272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f935cd526946779ecce236c9d47f3f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:02.763154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:02.808432 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f935cd526946779ecce236c9d47f3f', 'request_id': 'req-d274c77a-393a-4178-a738-30b372230f5a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.813923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f935cd526946779ecce236c9d47f3f', 'request_id': 'req-d274c77a-393a-4178-a738-30b372230f5a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.819878 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f935cd526946779ecce236c9d47f3f', 'request_id': 'req-d274c77a-393a-4178-a738-30b372230f5a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.831956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] The token cache is being invalidate because project d2f935cd526946779ecce236c9d47f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:05:02.832245 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.832867 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-d274c77a-393a-4178-a738-30b372230f5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:02.834389 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1394/2820] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:05:02 2022] DELETE /identity/v3/projects/d2f935cd526946779ecce236c9d47f3f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:02.839482 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8237e9-927c-4f24-ba17-fa99a4c0e710 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:02.839789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8237e9-927c-4f24-ba17-fa99a4c0e710 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:02.840065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8237e9-927c-4f24-ba17-fa99a4c0e710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:02.860387 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-3c8237e9-927c-4f24-ba17-fa99a4c0e710 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:02.861887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3c8237e9-927c-4f24-ba17-fa99a4c0e710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:02.919852 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1427/2821] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 02:05:02.928595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21acb5-3dc1-4e9d-91ac-5f1001221142 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:02.928880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21acb5-3dc1-4e9d-91ac-5f1001221142 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:02.929187 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21acb5-3dc1-4e9d-91ac-5f1001221142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:02.949484 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-fc21acb5-3dc1-4e9d-91ac-5f1001221142 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:02.950618 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-fc21acb5-3dc1-4e9d-91ac-5f1001221142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:02.958325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1395/2822] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:02.962453 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8237e9-927c-4f24-ba17-fa99a4c0e710 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:02.972472 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8237e9-927c-4f24-ba17-fa99a4c0e710 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:02.976980 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:02.977611 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:02.977888 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:02.978157 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:02.978792 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:05:02.979095 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:02.980696 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:02.982170 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:02.995065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1428/2823] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:05:02.998964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21acb5-3dc1-4e9d-91ac-5f1001221142 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.008080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21acb5-3dc1-4e9d-91ac-5f1001221142 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.012649 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b799-bf57-4a2a-8c8e-78cf0615e196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.013268 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b799-bf57-4a2a-8c8e-78cf0615e196 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.013545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b799-bf57-4a2a-8c8e-78cf0615e196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.013815 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1b799-bf57-4a2a-8c8e-78cf0615e196 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.014444 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1b799-bf57-4a2a-8c8e-78cf0615e196 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.016317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1b799-bf57-4a2a-8c8e-78cf0615e196 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.027175 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:03.044732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1396/2824] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:02 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:03.049408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.059265 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489665e-41bd-45b0-b16a-f37fe648ac75 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.115389 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7abbf-6069-4737-ba66-d82dbff66533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.115974 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7abbf-6069-4737-ba66-d82dbff66533 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.116253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7abbf-6069-4737-ba66-d82dbff66533 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.116527 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7abbf-6069-4737-ba66-d82dbff66533 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.117177 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7abbf-6069-4737-ba66-d82dbff66533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.119074 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7abbf-6069-4737-ba66-d82dbff66533 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.139766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1429/2825] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:03 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:03.144423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b799-bf57-4a2a-8c8e-78cf0615e196 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.154558 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1b799-bf57-4a2a-8c8e-78cf0615e196 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.159611 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41845d-be13-4322-a4da-f0bfdf842ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.160200 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41845d-be13-4322-a4da-f0bfdf842ccb admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.160478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41845d-be13-4322-a4da-f0bfdf842ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.160746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41845d-be13-4322-a4da-f0bfdf842ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.161414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41845d-be13-4322-a4da-f0bfdf842ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.162991 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41845d-be13-4322-a4da-f0bfdf842ccb admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.172928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1397/2826] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:03.176581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7abbf-6069-4737-ba66-d82dbff66533 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.186536 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7abbf-6069-4737-ba66-d82dbff66533 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.190981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887dae85-982d-4fd4-8f04-4783a39b3f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.191561 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dae85-982d-4fd4-8f04-4783a39b3f56 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.191836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dae85-982d-4fd4-8f04-4783a39b3f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.192104 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887dae85-982d-4fd4-8f04-4783a39b3f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.192759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887dae85-982d-4fd4-8f04-4783a39b3f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.194346 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887dae85-982d-4fd4-8f04-4783a39b3f56 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.205138 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1430/2827] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:03.209244 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41845d-be13-4322-a4da-f0bfdf842ccb admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.219577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41845d-be13-4322-a4da-f0bfdf842ccb admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.224154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921895-b987-45d0-90f3-bc5cb7e686d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.224784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921895-b987-45d0-90f3-bc5cb7e686d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.225067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921895-b987-45d0-90f3-bc5cb7e686d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.225350 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce921895-b987-45d0-90f3-bc5cb7e686d3 admin admin] PATH_INFO: `/v3/projects/1bc57f5cf4614db5aefe3538e79ff438/users/9dbdd44bf7994f368e4a567c8ae3c9d3/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.239591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce921895-b987-45d0-90f3-bc5cb7e686d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc57f5cf4614db5aefe3538e79ff438, user_id=9dbdd44bf7994f368e4a567c8ae3c9d3, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.242204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce921895-b987-45d0-90f3-bc5cb7e686d3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.254740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1398/2828] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:03 2022] PUT /identity/v3/projects/1bc57f5cf4614db5aefe3538e79ff438/users/9dbdd44bf7994f368e4a567c8ae3c9d3/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:03.265382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcb0e9-ff7b-49b8-9ee4-95e8e3beadcf None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.265382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcb0e9-ff7b-49b8-9ee4-95e8e3beadcf None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.267632 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcb0e9-ff7b-49b8-9ee4-95e8e3beadcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.293767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-abbcb0e9-ff7b-49b8-9ee4-95e8e3beadcf None None] MFA Rules not processed for user `9dbdd44bf7994f368e4a567c8ae3c9d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:03.294967 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-abbcb0e9-ff7b-49b8-9ee4-95e8e3beadcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:03.357981 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1431/2829] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:03.363345 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921895-b987-45d0-90f3-bc5cb7e686d3 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.424729 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce921895-b987-45d0-90f3-bc5cb7e686d3 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.429356 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024d3e5-e54b-4f03-a43a-561485a22a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.429956 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024d3e5-e54b-4f03-a43a-561485a22a4f admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.430236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024d3e5-e54b-4f03-a43a-561485a22a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.430513 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024d3e5-e54b-4f03-a43a-561485a22a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.431138 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024d3e5-e54b-4f03-a43a-561485a22a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.433009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024d3e5-e54b-4f03-a43a-561485a22a4f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.458866 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1399/2830] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:03 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:03.462945 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcb0e9-ff7b-49b8-9ee4-95e8e3beadcf None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.474073 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcb0e9-ff7b-49b8-9ee4-95e8e3beadcf None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.528767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69687e22-da7f-40e0-87d2-6319ea718f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.529394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687e22-da7f-40e0-87d2-6319ea718f94 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.529672 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687e22-da7f-40e0-87d2-6319ea718f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.529940 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69687e22-da7f-40e0-87d2-6319ea718f94 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.530545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69687e22-da7f-40e0-87d2-6319ea718f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.532913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69687e22-da7f-40e0-87d2-6319ea718f94 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.541567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:03.554142 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1432/2831] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:03 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:03.558026 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024d3e5-e54b-4f03-a43a-561485a22a4f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.568088 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024d3e5-e54b-4f03-a43a-561485a22a4f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.572701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44db8-a2ad-4009-a296-a828aada24f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.573311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44db8-a2ad-4009-a296-a828aada24f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.573595 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44db8-a2ad-4009-a296-a828aada24f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.573865 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44db8-a2ad-4009-a296-a828aada24f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.574495 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e44db8-a2ad-4009-a296-a828aada24f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.576067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e44db8-a2ad-4009-a296-a828aada24f8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.586285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1400/2832] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 02:05:03.590480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69687e22-da7f-40e0-87d2-6319ea718f94 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.600772 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69687e22-da7f-40e0-87d2-6319ea718f94 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.605365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319d684-078f-433b-a7ee-d9996751c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.606014 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319d684-078f-433b-a7ee-d9996751c276 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.606306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319d684-078f-433b-a7ee-d9996751c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.606556 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319d684-078f-433b-a7ee-d9996751c276 admin admin] PATH_INFO: `/v3/projects/ec12cbf8545349ec92a8597a7c2ac9e1/users/e8ae3b2b781b447682a0f74031cbd722/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.624726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319d684-078f-433b-a7ee-d9996751c276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec12cbf8545349ec92a8597a7c2ac9e1, user_id=e8ae3b2b781b447682a0f74031cbd722, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.627878 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319d684-078f-433b-a7ee-d9996751c276 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.642244 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1433/2833] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:03 2022] PUT /identity/v3/projects/ec12cbf8545349ec92a8597a7c2ac9e1/users/e8ae3b2b781b447682a0f74031cbd722/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:03.646001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44db8-a2ad-4009-a296-a828aada24f8 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.656057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44db8-a2ad-4009-a296-a828aada24f8 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.713906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932967a-bbbb-4c0e-bb8c-827a323dd67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.714447 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932967a-bbbb-4c0e-bb8c-827a323dd67b admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.714725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932967a-bbbb-4c0e-bb8c-827a323dd67b admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.714989 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932967a-bbbb-4c0e-bb8c-827a323dd67b admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.715609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932967a-bbbb-4c0e-bb8c-827a323dd67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.717208 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932967a-bbbb-4c0e-bb8c-827a323dd67b admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.727259 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1401/2834] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:03.731641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319d684-078f-433b-a7ee-d9996751c276 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.742770 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319d684-078f-433b-a7ee-d9996751c276 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.747343 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.747977 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.748253 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.748522 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] PATH_INFO: `/v3/projects/ec12cbf8545349ec92a8597a7c2ac9e1/users/e8ae3b2b781b447682a0f74031cbd722/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.750449 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec12cbf8545349ec92a8597a7c2ac9e1, user_id=e8ae3b2b781b447682a0f74031cbd722, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.753141 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.763115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1434/2835] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:03 2022] PUT /identity/v3/projects/ec12cbf8545349ec92a8597a7c2ac9e1/users/e8ae3b2b781b447682a0f74031cbd722/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:03.774392 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f013244-195c-45f5-b25d-834b46da00a3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.774680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f013244-195c-45f5-b25d-834b46da00a3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.774950 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f013244-195c-45f5-b25d-834b46da00a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.803446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6f013244-195c-45f5-b25d-834b46da00a3 None None] MFA Rules not processed for user `e8ae3b2b781b447682a0f74031cbd722`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:03.804651 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6f013244-195c-45f5-b25d-834b46da00a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:03.817692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:03.873892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1402/2836] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:03.885210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:03.886935 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:03.943385 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25834129-513e-4311-a895-fe9afcf6fd14 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:03.946983 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d4a88-18b8-4092-8dae-d2df326982c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:03.947584 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d4a88-18b8-4092-8dae-d2df326982c7 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:03.947862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d4a88-18b8-4092-8dae-d2df326982c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:03.948132 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d4a88-18b8-4092-8dae-d2df326982c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:03.957356 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d4a88-18b8-4092-8dae-d2df326982c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:03.958761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d4a88-18b8-4092-8dae-d2df326982c7 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:03.968002 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1435/2837] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:04.046293 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f013244-195c-45f5-b25d-834b46da00a3 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:04.055760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f013244-195c-45f5-b25d-834b46da00a3 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:04.059322 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f86054-2d0a-458b-bc31-9c8548bb3d25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:04.059909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f86054-2d0a-458b-bc31-9c8548bb3d25 service cinder] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:04.060185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f86054-2d0a-458b-bc31-9c8548bb3d25 service cinder] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:04.060473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f86054-2d0a-458b-bc31-9c8548bb3d25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:04.115906 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f86054-2d0a-458b-bc31-9c8548bb3d25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:04.117357 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f86054-2d0a-458b-bc31-9c8548bb3d25 service cinder] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:04.126194 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1403/2838] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:06.835552 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d4a88-18b8-4092-8dae-d2df326982c7 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:06.896371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d4a88-18b8-4092-8dae-d2df326982c7 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:06.900920 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:06.901587 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:06.901862 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:06.902129 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] PATH_INFO: `/v3/users/9dbdd44bf7994f368e4a567c8ae3c9d3` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:06.903223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dbdd44bf7994f368e4a567c8ae3c9d3)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:06.905131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:06.953733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dbdd44bf7994f368e4a567c8ae3c9d3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:06.954396 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dbdd44bf7994f368e4a567c8ae3c9d3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:06.962227 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:06.967499 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dbdd44bf7994f368e4a567c8ae3c9d3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:06.975827 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dbdd44bf7994f368e4a567c8ae3c9d3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:06.976408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dbdd44bf7994f368e4a567c8ae3c9d3'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:06.988819 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1436/2839] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:05:06 2022] DELETE /identity/v3/users/9dbdd44bf7994f368e4a567c8ae3c9d3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:06.993257 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31aa9-6aec-4a52-b7ef-d25c872071ab None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:06.993738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31aa9-6aec-4a52-b7ef-d25c872071ab None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:06.993852 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31aa9-6aec-4a52-b7ef-d25c872071ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:07.012750 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-57e31aa9-6aec-4a52-b7ef-d25c872071ab None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:07.013942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-57e31aa9-6aec-4a52-b7ef-d25c872071ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:07.070272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1404/2840] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:07.080347 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:07.081771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:07.137340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941bd82-2bf2-47c5-86b9-230f27cd8379 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:07.141394 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26cddb-0154-497f-9394-4c6b370219ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:07.141996 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26cddb-0154-497f-9394-4c6b370219ba service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:07.142264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26cddb-0154-497f-9394-4c6b370219ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:07.142531 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26cddb-0154-497f-9394-4c6b370219ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:07.151162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26cddb-0154-497f-9394-4c6b370219ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:07.152560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26cddb-0154-497f-9394-4c6b370219ba service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:07.162401 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1437/2841] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:07.436966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e31aa9-6aec-4a52-b7ef-d25c872071ab None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:07.439080 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-57e31aa9-6aec-4a52-b7ef-d25c872071ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:07.450998 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e31aa9-6aec-4a52-b7ef-d25c872071ab None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:07.456089 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:07.456730 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:07.456996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:07.457301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] PATH_INFO: `/v3/projects/1bc57f5cf4614db5aefe3538e79ff438` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:07.458418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc57f5cf4614db5aefe3538e79ff438)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:07.460317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:07.503609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:07.506679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc57f5cf4614db5aefe3538e79ff438', 'request_id': 'req-9634d612-63bb-4a27-b386-678c0dc49d1e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.512682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc57f5cf4614db5aefe3538e79ff438', 'request_id': 'req-9634d612-63bb-4a27-b386-678c0dc49d1e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.518871 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc57f5cf4614db5aefe3538e79ff438', 'request_id': 'req-9634d612-63bb-4a27-b386-678c0dc49d1e'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.530926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] The token cache is being invalidate because project 1bc57f5cf4614db5aefe3538e79ff438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:05:07.531731 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.532347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.533838 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1405/2842] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:05:07 2022] DELETE /identity/v3/projects/1bc57f5cf4614db5aefe3538e79ff438 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:07.538023 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26cddb-0154-497f-9394-4c6b370219ba service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:07.547887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26cddb-0154-497f-9394-4c6b370219ba service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:07.602167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:07.602794 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:07.603055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:07.603322 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] PATH_INFO: `/v3/users/e8ae3b2b781b447682a0f74031cbd722` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:07.604451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ae3b2b781b447682a0f74031cbd722)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:07.606351 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:07.650956 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ae3b2b781b447682a0f74031cbd722'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.651597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ae3b2b781b447682a0f74031cbd722'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.663607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ae3b2b781b447682a0f74031cbd722'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.671418 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ae3b2b781b447682a0f74031cbd722'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.672194 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-df53f0e3-ac64-4624-8e93-0d4050a51360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ae3b2b781b447682a0f74031cbd722'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:07.680085 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:07.685726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1438/2843] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:05:07 2022] DELETE /identity/v3/users/e8ae3b2b781b447682a0f74031cbd722 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:07.964522 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:07.966071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:07.980204 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634d612-63bb-4a27-b386-678c0dc49d1e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.037650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.038290 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.038517 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.038782 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] PATH_INFO: `/v3/projects/ec12cbf8545349ec92a8597a7c2ac9e1` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.039902 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec12cbf8545349ec92a8597a7c2ac9e1)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.042166 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.087490 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec12cbf8545349ec92a8597a7c2ac9e1', 'request_id': 'req-b643f2be-ab33-4e65-adab-0509f0d2337a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:08.093116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec12cbf8545349ec92a8597a7c2ac9e1', 'request_id': 'req-b643f2be-ab33-4e65-adab-0509f0d2337a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:08.099148 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec12cbf8545349ec92a8597a7c2ac9e1', 'request_id': 'req-b643f2be-ab33-4e65-adab-0509f0d2337a'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:08.111339 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] The token cache is being invalidate because project ec12cbf8545349ec92a8597a7c2ac9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:05:08.111628 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:08.112253 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-b643f2be-ab33-4e65-adab-0509f0d2337a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:08.113742 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1406/2844] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:05:07 2022] DELETE /identity/v3/projects/ec12cbf8545349ec92a8597a7c2ac9e1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:08.118557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2129-1112-4699-9516-e4973c802614 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.119044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2129-1112-4699-9516-e4973c802614 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.119158 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f2129-1112-4699-9516-e4973c802614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.138648 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ef9f2129-1112-4699-9516-e4973c802614 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:08.139841 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ef9f2129-1112-4699-9516-e4973c802614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:08.197984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1439/2845] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:08.205724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5659-57ec-4998-a8a3-933712d6de29 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.206012 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5659-57ec-4998-a8a3-933712d6de29 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.206287 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5659-57ec-4998-a8a3-933712d6de29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.225930 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0e9b5659-57ec-4998-a8a3-933712d6de29 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:08.227147 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0e9b5659-57ec-4998-a8a3-933712d6de29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:08.236529 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1407/2846] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:08.240750 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f2129-1112-4699-9516-e4973c802614 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.250327 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f2129-1112-4699-9516-e4973c802614 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.254835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.255402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.255675 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.255942 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] PATH_INFO: `/v3/domains` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.256567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87883) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:05:08.256864 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.258492 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.269570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1440/2847] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:05:08.273872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b5659-57ec-4998-a8a3-933712d6de29 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.283638 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b5659-57ec-4998-a8a3-933712d6de29 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.288325 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7566-1ae2-41eb-9674-d75eaf6c2ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.288854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7566-1ae2-41eb-9674-d75eaf6c2ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.289168 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7566-1ae2-41eb-9674-d75eaf6c2ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.289435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c7566-1ae2-41eb-9674-d75eaf6c2ff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.290049 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c7566-1ae2-41eb-9674-d75eaf6c2ff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.291909 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c7566-1ae2-41eb-9674-d75eaf6c2ff9 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.318533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1408/2848] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:08 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:08.322791 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.332953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b5fcb-c23f-4a6d-bff8-3e2cdca2f29c admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.389082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64f05c-098d-464f-a7a3-13bbfabcecf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.389690 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64f05c-098d-464f-a7a3-13bbfabcecf4 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.389995 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64f05c-098d-464f-a7a3-13bbfabcecf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.390264 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c64f05c-098d-464f-a7a3-13bbfabcecf4 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.391446 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64f05c-098d-464f-a7a3-13bbfabcecf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.393420 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c64f05c-098d-464f-a7a3-13bbfabcecf4 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.413775 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1441/2849] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:08 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:08.417957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7566-1ae2-41eb-9674-d75eaf6c2ff9 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.428093 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c7566-1ae2-41eb-9674-d75eaf6c2ff9 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.432680 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201594-5b88-4c77-9276-10be00309e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.433295 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201594-5b88-4c77-9276-10be00309e45 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.433572 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201594-5b88-4c77-9276-10be00309e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.433844 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201594-5b88-4c77-9276-10be00309e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.434477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201594-5b88-4c77-9276-10be00309e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.436045 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201594-5b88-4c77-9276-10be00309e45 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.446094 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1409/2850] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:08.449720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64f05c-098d-464f-a7a3-13bbfabcecf4 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.460127 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c64f05c-098d-464f-a7a3-13bbfabcecf4 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.464836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631983a2-6349-416d-9f97-0f744eec3a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.465458 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631983a2-6349-416d-9f97-0f744eec3a1a admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.465736 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631983a2-6349-416d-9f97-0f744eec3a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.466032 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631983a2-6349-416d-9f97-0f744eec3a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.466661 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631983a2-6349-416d-9f97-0f744eec3a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.468243 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631983a2-6349-416d-9f97-0f744eec3a1a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.479199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1442/2851] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:08.483038 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201594-5b88-4c77-9276-10be00309e45 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.492946 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201594-5b88-4c77-9276-10be00309e45 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.497439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65562d08-b995-4441-be5e-45ea68851133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.498069 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65562d08-b995-4441-be5e-45ea68851133 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.498344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65562d08-b995-4441-be5e-45ea68851133 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.498612 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65562d08-b995-4441-be5e-45ea68851133 admin admin] PATH_INFO: `/v3/projects/7900ec63a0fc45a09ec6fc9e29a178d4/users/b3c60198637a48f0baed9c92ef9868eb/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.512789 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65562d08-b995-4441-be5e-45ea68851133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7900ec63a0fc45a09ec6fc9e29a178d4, user_id=b3c60198637a48f0baed9c92ef9868eb, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.515904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65562d08-b995-4441-be5e-45ea68851133 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.524197 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:08.529526 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1410/2852] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:08 2022] PUT /identity/v3/projects/7900ec63a0fc45a09ec6fc9e29a178d4/users/b3c60198637a48f0baed9c92ef9868eb/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:08.541267 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b333df9-d362-4895-bab0-4a6c96ab55b4 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.541529 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b333df9-d362-4895-bab0-4a6c96ab55b4 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.541779 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b333df9-d362-4895-bab0-4a6c96ab55b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.571483 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-3b333df9-d362-4895-bab0-4a6c96ab55b4 None None] MFA Rules not processed for user `b3c60198637a48f0baed9c92ef9868eb`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:08.572689 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-3b333df9-d362-4895-bab0-4a6c96ab55b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:08.626625 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:08.637708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1443/2853] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:08.643014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65562d08-b995-4441-be5e-45ea68851133 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.703508 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65562d08-b995-4441-be5e-45ea68851133 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.708170 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96839ae3-d980-48fb-b6c5-14ccedff2edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.708764 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96839ae3-d980-48fb-b6c5-14ccedff2edd admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.709044 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96839ae3-d980-48fb-b6c5-14ccedff2edd admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.709346 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96839ae3-d980-48fb-b6c5-14ccedff2edd admin admin] PATH_INFO: `/v3/projects` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.709973 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96839ae3-d980-48fb-b6c5-14ccedff2edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.711855 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96839ae3-d980-48fb-b6c5-14ccedff2edd admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.737829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1411/2854] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:08 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:08.741785 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b333df9-d362-4895-bab0-4a6c96ab55b4 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.752771 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b333df9-d362-4895-bab0-4a6c96ab55b4 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.807928 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bbfb-7566-48f1-9f2a-d8fe325ab8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.808461 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bbfb-7566-48f1-9f2a-d8fe325ab8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.808740 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bbfb-7566-48f1-9f2a-d8fe325ab8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.809011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8bbfb-7566-48f1-9f2a-d8fe325ab8d0 admin admin] PATH_INFO: `/v3/users` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.809668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8bbfb-7566-48f1-9f2a-d8fe325ab8d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.811560 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8bbfb-7566-48f1-9f2a-d8fe325ab8d0 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.833053 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1444/2855] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:08 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:08.837040 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96839ae3-d980-48fb-b6c5-14ccedff2edd admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.847308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96839ae3-d980-48fb-b6c5-14ccedff2edd admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.852078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e375-9f0e-4e26-9068-648a768ab8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.852959 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e375-9f0e-4e26-9068-648a768ab8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.853281 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e375-9f0e-4e26-9068-648a768ab8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.853551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3e375-9f0e-4e26-9068-648a768ab8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.854181 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3e375-9f0e-4e26-9068-648a768ab8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.855747 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3e375-9f0e-4e26-9068-648a768ab8c8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.866376 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1412/2856] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 02:05:08.870365 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bbfb-7566-48f1-9f2a-d8fe325ab8d0 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.880826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8bbfb-7566-48f1-9f2a-d8fe325ab8d0 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.885516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa38c87-87ce-4423-9386-c433b642670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.886184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa38c87-87ce-4423-9386-c433b642670f admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.886488 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa38c87-87ce-4423-9386-c433b642670f admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.886761 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa38c87-87ce-4423-9386-c433b642670f admin admin] PATH_INFO: `/v3/projects/14f818a4fa974301b0e407fa07f90fe5/users/494ec99a2be44ccab52686e76c40383f/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.902187 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa38c87-87ce-4423-9386-c433b642670f admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f818a4fa974301b0e407fa07f90fe5, user_id=494ec99a2be44ccab52686e76c40383f, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.904826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa38c87-87ce-4423-9386-c433b642670f admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:08.921218 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1445/2857] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:08 2022] PUT /identity/v3/projects/14f818a4fa974301b0e407fa07f90fe5/users/494ec99a2be44ccab52686e76c40383f/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:08.925225 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e375-9f0e-4e26-9068-648a768ab8c8 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:08.935056 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3e375-9f0e-4e26-9068-648a768ab8c8 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:08.990236 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b776c-a87e-44ed-88b0-65886e59f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:08.990845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b776c-a87e-44ed-88b0-65886e59f0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:08.991130 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b776c-a87e-44ed-88b0-65886e59f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:08.991402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b776c-a87e-44ed-88b0-65886e59f0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:08.992032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b776c-a87e-44ed-88b0-65886e59f0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:08.993689 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b776c-a87e-44ed-88b0-65886e59f0c3 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:09.003977 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1413/2858] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:09.008416 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa38c87-87ce-4423-9386-c433b642670f admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:09.019198 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa38c87-87ce-4423-9386-c433b642670f admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:09.023784 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:09.024454 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:09.024718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:09.024993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] PATH_INFO: `/v3/projects/14f818a4fa974301b0e407fa07f90fe5/users/494ec99a2be44ccab52686e76c40383f/roles/b131b61299154544a627315649090a49` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:09.026938 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f818a4fa974301b0e407fa07f90fe5, user_id=494ec99a2be44ccab52686e76c40383f, role_id=b131b61299154544a627315649090a49)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:09.029597 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:09.039558 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1446/2859] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:09 2022] PUT /identity/v3/projects/14f818a4fa974301b0e407fa07f90fe5/users/494ec99a2be44ccab52686e76c40383f/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:09.100793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf55ff-83af-4e6d-acb5-8e5183b4a172 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:09.101418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf55ff-83af-4e6d-acb5-8e5183b4a172 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:09.101418 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf55ff-83af-4e6d-acb5-8e5183b4a172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:09.130133 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-85cf55ff-83af-4e6d-acb5-8e5183b4a172 None None] MFA Rules not processed for user `494ec99a2be44ccab52686e76c40383f`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:09.131335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-85cf55ff-83af-4e6d-acb5-8e5183b4a172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:09.188443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:09.197985 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1414/2860] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:09.208885 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:09.210703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:09.267388 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae4241-24e0-4658-b433-d581bc1b01f3 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:09.271098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936a27a-d4a3-43a7-81cb-9e88a7c032ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:09.271693 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936a27a-d4a3-43a7-81cb-9e88a7c032ad service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:09.271970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936a27a-d4a3-43a7-81cb-9e88a7c032ad service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:09.272241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936a27a-d4a3-43a7-81cb-9e88a7c032ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:09.281035 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936a27a-d4a3-43a7-81cb-9e88a7c032ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:09.282609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a936a27a-d4a3-43a7-81cb-9e88a7c032ad service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:09.291944 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1447/2861] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:10.627459 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf55ff-83af-4e6d-acb5-8e5183b4a172 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:10.688929 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf55ff-83af-4e6d-acb5-8e5183b4a172 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:10.693509 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:10.694128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:10.694408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:10.694682 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] PATH_INFO: `/v3/users/b3c60198637a48f0baed9c92ef9868eb` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:10.695797 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c60198637a48f0baed9c92ef9868eb)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:10.698009 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:10.752694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c60198637a48f0baed9c92ef9868eb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:10.753381 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c60198637a48f0baed9c92ef9868eb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:10.760609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c60198637a48f0baed9c92ef9868eb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:10.772732 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c60198637a48f0baed9c92ef9868eb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:10.779674 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c60198637a48f0baed9c92ef9868eb'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:10.785968 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1415/2862] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:05:10 2022] DELETE /identity/v3/users/b3c60198637a48f0baed9c92ef9868eb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:10.790430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a216f7f-49b2-424a-8edf-b91883b595b0 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:10.790918 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a216f7f-49b2-424a-8edf-b91883b595b0 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:10.791025 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a216f7f-49b2-424a-8edf-b91883b595b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:10.809601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5a216f7f-49b2-424a-8edf-b91883b595b0 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:10.810774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5a216f7f-49b2-424a-8edf-b91883b595b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:10.853836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:10.870181 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1448/2863] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 02:05:10.880301 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:10.881725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:10.939715 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26478caa-6d13-4d4e-9ee5-2b1784199b3e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:10.943407 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327bdae-5a65-4816-b526-2196f2ea0dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:10.944004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327bdae-5a65-4816-b526-2196f2ea0dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:10.944280 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327bdae-5a65-4816-b526-2196f2ea0dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:10.944549 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327bdae-5a65-4816-b526-2196f2ea0dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:10.953467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327bdae-5a65-4816-b526-2196f2ea0dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:10.954877 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327bdae-5a65-4816-b526-2196f2ea0dd3 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:10.964074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1416/2864] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:11.224943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a216f7f-49b2-424a-8edf-b91883b595b0 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:11.226381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5a216f7f-49b2-424a-8edf-b91883b595b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:11.239055 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a216f7f-49b2-424a-8edf-b91883b595b0 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:11.243676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:11.244304 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:11.244576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:11.244849 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] PATH_INFO: `/v3/projects/7900ec63a0fc45a09ec6fc9e29a178d4` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:11.245985 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7900ec63a0fc45a09ec6fc9e29a178d4)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:11.247867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:11.293917 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7900ec63a0fc45a09ec6fc9e29a178d4', 'request_id': 'req-6da815c8-dc92-485d-9d59-90699d4be54e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.300570 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7900ec63a0fc45a09ec6fc9e29a178d4', 'request_id': 'req-6da815c8-dc92-485d-9d59-90699d4be54e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.306016 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7900ec63a0fc45a09ec6fc9e29a178d4', 'request_id': 'req-6da815c8-dc92-485d-9d59-90699d4be54e'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.317829 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] The token cache is being invalidate because project 7900ec63a0fc45a09ec6fc9e29a178d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:05:11.318115 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.318734 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.320207 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1449/2865] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:05:11 2022] DELETE /identity/v3/projects/7900ec63a0fc45a09ec6fc9e29a178d4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:11.324306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327bdae-5a65-4816-b526-2196f2ea0dd3 service neutron] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:11.335185 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327bdae-5a65-4816-b526-2196f2ea0dd3 service neutron] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:11.347778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:11.396465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:11.397207 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:11.397537 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:11.397926 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] PATH_INFO: `/v3/users/494ec99a2be44ccab52686e76c40383f` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:11.399485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=494ec99a2be44ccab52686e76c40383f)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:11.401933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:11.447313 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '494ec99a2be44ccab52686e76c40383f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.447955 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '494ec99a2be44ccab52686e76c40383f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.456512 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '494ec99a2be44ccab52686e76c40383f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.469583 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '494ec99a2be44ccab52686e76c40383f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.476300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-921a22c0-61af-4229-8fd6-de736900a926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '494ec99a2be44ccab52686e76c40383f'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.482760 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1417/2866] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:05:11 2022] DELETE /identity/v3/users/494ec99a2be44ccab52686e76c40383f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:05:11.745516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:11.747026 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:11.768414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da815c8-dc92-485d-9d59-90699d4be54e admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:11.823559 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:11.824248 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:11.824441 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:11.824711 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] PATH_INFO: `/v3/projects/14f818a4fa974301b0e407fa07f90fe5` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:11.825838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f818a4fa974301b0e407fa07f90fe5)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:11.827856 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:11.872078 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f818a4fa974301b0e407fa07f90fe5', 'request_id': 'req-e961fa6d-0b44-46ad-b9f2-fd9512712c32'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.878621 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f818a4fa974301b0e407fa07f90fe5', 'request_id': 'req-e961fa6d-0b44-46ad-b9f2-fd9512712c32'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.884110 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f818a4fa974301b0e407fa07f90fe5', 'request_id': 'req-e961fa6d-0b44-46ad-b9f2-fd9512712c32'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.896898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] The token cache is being invalidate because project 14f818a4fa974301b0e407fa07f90fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:05:11.897252 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.897905 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-e961fa6d-0b44-46ad-b9f2-fd9512712c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:11.899432 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1450/2867] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:05:11 2022] DELETE /identity/v3/projects/14f818a4fa974301b0e407fa07f90fe5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:11.904398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd03f5e-92d3-4c46-93dd-987153370311 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:11.904876 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd03f5e-92d3-4c46-93dd-987153370311 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:11.904996 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd03f5e-92d3-4c46-93dd-987153370311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:11.924150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-0bd03f5e-92d3-4c46-93dd-987153370311 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:11.925269 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-0bd03f5e-92d3-4c46-93dd-987153370311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:11.982013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1418/2868] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:11.990408 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5734b-7d6f-4d41-8843-0581c2b03090 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:11.990677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5734b-7d6f-4d41-8843-0581c2b03090 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:11.990953 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5734b-7d6f-4d41-8843-0581c2b03090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:11.998764 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:12.012820 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-aed5734b-7d6f-4d41-8843-0581c2b03090 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:12.014044 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-aed5734b-7d6f-4d41-8843-0581c2b03090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:12.022034 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1451/2869] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:12.026299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd03f5e-92d3-4c46-93dd-987153370311 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.036344 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd03f5e-92d3-4c46-93dd-987153370311 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.040943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.041565 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.041843 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.042113 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.042778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:05:12.043086 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.044664 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.056508 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1419/2870] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:05:12.060608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5734b-7d6f-4d41-8843-0581c2b03090 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.070124 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5734b-7d6f-4d41-8843-0581c2b03090 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.074774 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d889f-ec3e-4159-a402-54cb886e0bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.075363 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d889f-ec3e-4159-a402-54cb886e0bbd admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.075637 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d889f-ec3e-4159-a402-54cb886e0bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.075907 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d889f-ec3e-4159-a402-54cb886e0bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.076537 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d889f-ec3e-4159-a402-54cb886e0bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.079101 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d889f-ec3e-4159-a402-54cb886e0bbd admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.107201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1452/2871] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:12 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:12.111817 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.121665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4bd3e-997d-4866-9da1-4e5545b4581f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.142154 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:12.179231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f6907-edba-4d57-83f2-a882cd6487dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.179756 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6907-edba-4d57-83f2-a882cd6487dc admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.180032 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6907-edba-4d57-83f2-a882cd6487dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.180299 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f6907-edba-4d57-83f2-a882cd6487dc admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.180917 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f6907-edba-4d57-83f2-a882cd6487dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.182942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f6907-edba-4d57-83f2-a882cd6487dc admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.203487 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1420/2872] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:12 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:12.207607 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d889f-ec3e-4159-a402-54cb886e0bbd admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.217667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d889f-ec3e-4159-a402-54cb886e0bbd admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.222355 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2bdf9-b798-4e7d-b62d-47e3e9c5b620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.222925 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2bdf9-b798-4e7d-b62d-47e3e9c5b620 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.223199 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2bdf9-b798-4e7d-b62d-47e3e9c5b620 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.223464 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2bdf9-b798-4e7d-b62d-47e3e9c5b620 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.224098 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2bdf9-b798-4e7d-b62d-47e3e9c5b620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.225692 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2bdf9-b798-4e7d-b62d-47e3e9c5b620 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.235857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1453/2873] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:12.239819 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f6907-edba-4d57-83f2-a882cd6487dc admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.249573 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f6907-edba-4d57-83f2-a882cd6487dc admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.254672 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693f6e0-f342-4b35-9fda-d1f747c25216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.255248 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f6e0-f342-4b35-9fda-d1f747c25216 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.255523 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f6e0-f342-4b35-9fda-d1f747c25216 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.255789 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693f6e0-f342-4b35-9fda-d1f747c25216 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.256414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f6e0-f342-4b35-9fda-d1f747c25216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.258261 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693f6e0-f342-4b35-9fda-d1f747c25216 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.269052 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1421/2874] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:12.272922 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2bdf9-b798-4e7d-b62d-47e3e9c5b620 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.282812 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2bdf9-b798-4e7d-b62d-47e3e9c5b620 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.287519 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.288162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.288439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.288708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] PATH_INFO: `/v3/projects/2579f574e977430d8d424431c9c2edbf/users/1005cacf53204833a2c70fc80c259cfc/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.304071 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2579f574e977430d8d424431c9c2edbf, user_id=1005cacf53204833a2c70fc80c259cfc, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.306727 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.319503 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1454/2875] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:12 2022] PUT /identity/v3/projects/2579f574e977430d8d424431c9c2edbf/users/1005cacf53204833a2c70fc80c259cfc/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:12.330727 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd9c42-1ab7-416c-8c2b-a76090a6ec50 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.331013 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd9c42-1ab7-416c-8c2b-a76090a6ec50 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.331286 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd9c42-1ab7-416c-8c2b-a76090a6ec50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.358721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-86dd9c42-1ab7-416c-8c2b-a76090a6ec50 None None] MFA Rules not processed for user `1005cacf53204833a2c70fc80c259cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:12.361458 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-86dd9c42-1ab7-416c-8c2b-a76090a6ec50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:12.426931 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1422/2876] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:12.438065 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.439733 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:12.497261 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0af323-c8fa-470c-becb-56d83231868d admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.500914 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3adbf-6ab9-47e1-9ccf-ed2c781e20c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.501557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3adbf-6ab9-47e1-9ccf-ed2c781e20c1 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.501835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3adbf-6ab9-47e1-9ccf-ed2c781e20c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.502105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3adbf-6ab9-47e1-9ccf-ed2c781e20c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.510381 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3adbf-6ab9-47e1-9ccf-ed2c781e20c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.511817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3adbf-6ab9-47e1-9ccf-ed2c781e20c1 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.520512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1455/2877] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:12.542163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd9c42-1ab7-416c-8c2b-a76090a6ec50 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.602879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd9c42-1ab7-416c-8c2b-a76090a6ec50 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.607422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.608031 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.608303 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.608570 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] PATH_INFO: `/v3/users/1005cacf53204833a2c70fc80c259cfc` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.610266 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1005cacf53204833a2c70fc80c259cfc)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.612435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.649579 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:12.661199 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1005cacf53204833a2c70fc80c259cfc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:12.661870 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1005cacf53204833a2c70fc80c259cfc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:12.673834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1005cacf53204833a2c70fc80c259cfc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:12.686465 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1005cacf53204833a2c70fc80c259cfc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:12.693767 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1005cacf53204833a2c70fc80c259cfc'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:12.700042 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1423/2878] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:05:12 2022] DELETE /identity/v3/users/1005cacf53204833a2c70fc80c259cfc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:12.705281 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309e588-509d-4b32-bf60-0fc78c075397 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.705580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309e588-509d-4b32-bf60-0fc78c075397 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.705924 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309e588-509d-4b32-bf60-0fc78c075397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.724763 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-e309e588-509d-4b32-bf60-0fc78c075397 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:12.725984 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e309e588-509d-4b32-bf60-0fc78c075397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:12.782890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1456/2879] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:12.792626 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:12.794074 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:12.850507 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c263f-63e2-4627-b857-d6b7ff93ce99 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:12.854190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8c86d-e1c7-41b1-93db-b6a8ac55807a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:12.854793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8c86d-e1c7-41b1-93db-b6a8ac55807a service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:12.855070 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8c86d-e1c7-41b1-93db-b6a8ac55807a service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:12.855340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8c86d-e1c7-41b1-93db-b6a8ac55807a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:12.864888 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8c86d-e1c7-41b1-93db-b6a8ac55807a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:12.866359 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8c86d-e1c7-41b1-93db-b6a8ac55807a service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:12.876255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1424/2880] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:13.141182 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309e588-509d-4b32-bf60-0fc78c075397 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.142732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-e309e588-509d-4b32-bf60-0fc78c075397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:13.154703 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309e588-509d-4b32-bf60-0fc78c075397 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.160352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.160982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.161306 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.161581 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] PATH_INFO: `/v3/projects/2579f574e977430d8d424431c9c2edbf` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.162701 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2579f574e977430d8d424431c9c2edbf)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.164613 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:13.202695 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:13.212232 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2579f574e977430d8d424431c9c2edbf', 'request_id': 'req-b7004e16-c5f6-4131-856d-1af736793154'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:13.219051 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2579f574e977430d8d424431c9c2edbf', 'request_id': 'req-b7004e16-c5f6-4131-856d-1af736793154'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:13.224601 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2579f574e977430d8d424431c9c2edbf', 'request_id': 'req-b7004e16-c5f6-4131-856d-1af736793154'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:13.237319 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] The token cache is being invalidate because project 2579f574e977430d8d424431c9c2edbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:05:13.237609 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:13.238249 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b7004e16-c5f6-4131-856d-1af736793154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:13.239934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1457/2881] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:05:13 2022] DELETE /identity/v3/projects/2579f574e977430d8d424431c9c2edbf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:13.244546 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2768bc-de95-4052-876c-84fc55bd3e77 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.245004 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2768bc-de95-4052-876c-84fc55bd3e77 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.245129 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2768bc-de95-4052-876c-84fc55bd3e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.265014 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-3d2768bc-de95-4052-876c-84fc55bd3e77 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:13.266222 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3d2768bc-de95-4052-876c-84fc55bd3e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:13.323323 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1425/2882] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:13.330937 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812487e-f598-4a20-b336-a91a626634be None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.331213 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812487e-f598-4a20-b336-a91a626634be None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.331484 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812487e-f598-4a20-b336-a91a626634be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.352333 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-2812487e-f598-4a20-b336-a91a626634be None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:13.353554 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-2812487e-f598-4a20-b336-a91a626634be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:13.362195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1458/2883] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:13.366643 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2768bc-de95-4052-876c-84fc55bd3e77 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.376240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2768bc-de95-4052-876c-84fc55bd3e77 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.380739 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.381414 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.381693 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.381964 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.382602 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:05:13.382901 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.384473 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:13.396126 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1426/2884] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:05:13.400923 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812487e-f598-4a20-b336-a91a626634be None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.410580 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812487e-f598-4a20-b336-a91a626634be None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.415215 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912fbce-d24a-414d-81e0-25171ac28774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.415801 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912fbce-d24a-414d-81e0-25171ac28774 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.416084 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912fbce-d24a-414d-81e0-25171ac28774 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.416354 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912fbce-d24a-414d-81e0-25171ac28774 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.416986 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912fbce-d24a-414d-81e0-25171ac28774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.418891 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912fbce-d24a-414d-81e0-25171ac28774 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:13.445378 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1459/2885] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:13 2022] POST /identity/v3/projects => generated 359 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:13.449402 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.459241 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a3883-0bbc-42d4-b49a-48c838be39a2 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.499318 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:13.517136 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb76db-ae90-477d-901a-006df72febe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.517738 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb76db-ae90-477d-901a-006df72febe5 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.518016 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb76db-ae90-477d-901a-006df72febe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.518285 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb76db-ae90-477d-901a-006df72febe5 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.518903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb76db-ae90-477d-901a-006df72febe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.520787 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb76db-ae90-477d-901a-006df72febe5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:13.541480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1427/2886] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:13 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:13.545628 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912fbce-d24a-414d-81e0-25171ac28774 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.556156 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912fbce-d24a-414d-81e0-25171ac28774 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.560593 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe9e44-d89c-41e9-9b2e-cda71b3eb2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.561206 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe9e44-d89c-41e9-9b2e-cda71b3eb2ad admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.561480 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe9e44-d89c-41e9-9b2e-cda71b3eb2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.561744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe9e44-d89c-41e9-9b2e-cda71b3eb2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.562371 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe9e44-d89c-41e9-9b2e-cda71b3eb2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.563930 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe9e44-d89c-41e9-9b2e-cda71b3eb2ad admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:13.573666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1460/2887] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:13.577317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb76db-ae90-477d-901a-006df72febe5 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.587500 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb76db-ae90-477d-901a-006df72febe5 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.592265 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a60d5-990b-4022-b770-d167211df935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.592849 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a60d5-990b-4022-b770-d167211df935 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.593162 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a60d5-990b-4022-b770-d167211df935 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.594033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a60d5-990b-4022-b770-d167211df935 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.594690 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a60d5-990b-4022-b770-d167211df935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.596251 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a60d5-990b-4022-b770-d167211df935 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:13.607001 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1428/2888] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:13.610676 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe9e44-d89c-41e9-9b2e-cda71b3eb2ad admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.620596 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe9e44-d89c-41e9-9b2e-cda71b3eb2ad admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.625114 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.625747 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.626020 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.626290 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] PATH_INFO: `/v3/projects/99aac5e315674b56b147954902fbe159/users/d7f877b136a9430b9a100142571a383d/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.640746 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99aac5e315674b56b147954902fbe159, user_id=d7f877b136a9430b9a100142571a383d, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.643512 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:13.655640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1461/2889] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:13 2022] PUT /identity/v3/projects/99aac5e315674b56b147954902fbe159/users/d7f877b136a9430b9a100142571a383d/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:13.666489 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122f7c7-57ea-4b72-8730-2f1c2630b088 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.666778 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122f7c7-57ea-4b72-8730-2f1c2630b088 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.667050 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122f7c7-57ea-4b72-8730-2f1c2630b088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.695475 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-3122f7c7-57ea-4b72-8730-2f1c2630b088 None None] MFA Rules not processed for user `d7f877b136a9430b9a100142571a383d`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:13.696834 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3122f7c7-57ea-4b72-8730-2f1c2630b088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:13.762698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1429/2890] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:13.797753 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.799456 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:13.856835 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91242322-6a41-45b7-998e-1794fd105fe5 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.860658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511fee6-eb8d-47e5-a24f-4ca192525dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.861542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511fee6-eb8d-47e5-a24f-4ca192525dff service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.861821 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511fee6-eb8d-47e5-a24f-4ca192525dff service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.862092 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511fee6-eb8d-47e5-a24f-4ca192525dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.870523 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511fee6-eb8d-47e5-a24f-4ca192525dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.872000 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511fee6-eb8d-47e5-a24f-4ca192525dff service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:13.880898 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1462/2891] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:13.897577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122f7c7-57ea-4b72-8730-2f1c2630b088 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:13.959853 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122f7c7-57ea-4b72-8730-2f1c2630b088 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:13.964426 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:13.965053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:13.965360 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:13.965634 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] PATH_INFO: `/v3/users/d7f877b136a9430b9a100142571a383d` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:13.966755 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7f877b136a9430b9a100142571a383d)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:13.968692 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:14.017937 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7f877b136a9430b9a100142571a383d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.018641 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7f877b136a9430b9a100142571a383d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.023805 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:14.027478 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7f877b136a9430b9a100142571a383d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.041897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7f877b136a9430b9a100142571a383d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.048657 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7f877b136a9430b9a100142571a383d'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.054614 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1430/2892] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:05:13 2022] DELETE /identity/v3/users/d7f877b136a9430b9a100142571a383d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:14.059197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958e501-6e5a-4806-b445-50ec1fbfe317 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.059504 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958e501-6e5a-4806-b445-50ec1fbfe317 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.059773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958e501-6e5a-4806-b445-50ec1fbfe317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.079226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4958e501-6e5a-4806-b445-50ec1fbfe317 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:14.080402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4958e501-6e5a-4806-b445-50ec1fbfe317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:14.137759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1463/2893] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:14.147988 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:14.149433 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:14.205578 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a78df-0c01-428b-88a4-b73bbeb6fe00 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:14.209554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baeb01d-3aff-435c-9790-7297fb361134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:14.210156 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeb01d-3aff-435c-9790-7297fb361134 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.210437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeb01d-3aff-435c-9790-7297fb361134 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.210710 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baeb01d-3aff-435c-9790-7297fb361134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.219688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baeb01d-3aff-435c-9790-7297fb361134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:14.221128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baeb01d-3aff-435c-9790-7297fb361134 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:14.230485 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1431/2894] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:14.497685 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958e501-6e5a-4806-b445-50ec1fbfe317 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:14.499266 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4958e501-6e5a-4806-b445-50ec1fbfe317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:14.507451 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:14.511358 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958e501-6e5a-4806-b445-50ec1fbfe317 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:14.518544 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:14.519409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.519847 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.520288 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] PATH_INFO: `/v3/projects/99aac5e315674b56b147954902fbe159` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.521542 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99aac5e315674b56b147954902fbe159)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:14.523517 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:14.573863 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99aac5e315674b56b147954902fbe159', 'request_id': 'req-bb622391-b61c-48a7-ae25-5966adea4df9'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.581155 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99aac5e315674b56b147954902fbe159', 'request_id': 'req-bb622391-b61c-48a7-ae25-5966adea4df9'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.587447 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99aac5e315674b56b147954902fbe159', 'request_id': 'req-bb622391-b61c-48a7-ae25-5966adea4df9'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.601363 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] The token cache is being invalidate because project 99aac5e315674b56b147954902fbe159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:05:14.601649 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.602430 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-bb622391-b61c-48a7-ae25-5966adea4df9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:14.603931 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1464/2895] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:05:14 2022] DELETE /identity/v3/projects/99aac5e315674b56b147954902fbe159 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:05:14.610386 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef22453-24c6-4350-94f8-6a9bc7c13d22 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.611226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef22453-24c6-4350-94f8-6a9bc7c13d22 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.611226 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef22453-24c6-4350-94f8-6a9bc7c13d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.630943 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-5ef22453-24c6-4350-94f8-6a9bc7c13d22 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:14.632144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5ef22453-24c6-4350-94f8-6a9bc7c13d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:14.700020 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1432/2896] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:14.709640 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807db8-c876-41e6-b261-59046b34c586 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.709927 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807db8-c876-41e6-b261-59046b34c586 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.710197 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa807db8-c876-41e6-b261-59046b34c586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.729854 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-aa807db8-c876-41e6-b261-59046b34c586 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:14.731027 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-aa807db8-c876-41e6-b261-59046b34c586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:14.739286 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1465/2897] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:14.743435 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef22453-24c6-4350-94f8-6a9bc7c13d22 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:14.753316 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef22453-24c6-4350-94f8-6a9bc7c13d22 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:14.757812 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:14.758398 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.758673 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.758940 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.759577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:05:14.759879 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:14.761477 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:14.772688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1433/2898] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:05:14.776828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa807db8-c876-41e6-b261-59046b34c586 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:14.786546 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa807db8-c876-41e6-b261-59046b34c586 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:14.791241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77d642-9736-468d-b1ee-754e93441cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:14.791828 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77d642-9736-468d-b1ee-754e93441cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.792105 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77d642-9736-468d-b1ee-754e93441cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.792374 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77d642-9736-468d-b1ee-754e93441cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.793003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77d642-9736-468d-b1ee-754e93441cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:14.794976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77d642-9736-468d-b1ee-754e93441cc7 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:14.821950 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1466/2899] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:14 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:14.825974 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:14.836349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05845250-0da6-4d4e-a24d-41aba4a41274 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:14.890892 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142685a0-9f66-4b4a-87ef-bb7cc40d4abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:14.891488 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142685a0-9f66-4b4a-87ef-bb7cc40d4abe admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.891784 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142685a0-9f66-4b4a-87ef-bb7cc40d4abe admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.892161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142685a0-9f66-4b4a-87ef-bb7cc40d4abe admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.892896 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142685a0-9f66-4b4a-87ef-bb7cc40d4abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:14.894860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142685a0-9f66-4b4a-87ef-bb7cc40d4abe admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:14.903053 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:14.917186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1434/2900] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:14 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:14.921363 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77d642-9736-468d-b1ee-754e93441cc7 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:14.931562 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77d642-9736-468d-b1ee-754e93441cc7 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:14.936269 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3ea6d-9ded-4482-ba72-bd50192c5c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:14.936857 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3ea6d-9ded-4482-ba72-bd50192c5c51 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.937660 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3ea6d-9ded-4482-ba72-bd50192c5c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.937933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d3ea6d-9ded-4482-ba72-bd50192c5c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.938567 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3ea6d-9ded-4482-ba72-bd50192c5c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:14.940169 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d3ea6d-9ded-4482-ba72-bd50192c5c51 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:14.949894 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1467/2901] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:14.954416 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142685a0-9f66-4b4a-87ef-bb7cc40d4abe admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:14.964141 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142685a0-9f66-4b4a-87ef-bb7cc40d4abe admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:14.968793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cb044-72a8-4c60-b613-f37f79d648c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:14.969419 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb044-72a8-4c60-b613-f37f79d648c5 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:14.969699 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb044-72a8-4c60-b613-f37f79d648c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:14.969970 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cb044-72a8-4c60-b613-f37f79d648c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:14.970600 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cb044-72a8-4c60-b613-f37f79d648c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:14.972173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cb044-72a8-4c60-b613-f37f79d648c5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:14.983594 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1435/2902] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:14.987875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3ea6d-9ded-4482-ba72-bd50192c5c51 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:14.997664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d3ea6d-9ded-4482-ba72-bd50192c5c51 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:15.002667 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:15.003297 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:15.003572 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:15.003838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] PATH_INFO: `/v3/projects/905d7c2123df411488e6a86e40a5b4a1/users/646bf439f58145c28acc5599057d51f3/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:15.018334 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=905d7c2123df411488e6a86e40a5b4a1, user_id=646bf439f58145c28acc5599057d51f3, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:15.020947 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:15.033759 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1468/2903] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:14 2022] PUT /identity/v3/projects/905d7c2123df411488e6a86e40a5b4a1/users/646bf439f58145c28acc5599057d51f3/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:05:15.044137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5a59-778e-4c00-8726-7b22f3a571f3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:15.044422 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5a59-778e-4c00-8726-7b22f3a571f3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:15.044694 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5a59-778e-4c00-8726-7b22f3a571f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:15.072503 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-64be5a59-778e-4c00-8726-7b22f3a571f3 None None] MFA Rules not processed for user `646bf439f58145c28acc5599057d51f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:15.073748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-64be5a59-778e-4c00-8726-7b22f3a571f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:15.136067 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1436/2904] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:15.148056 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:15.149766 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:15.205726 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40d79d-3d29-4574-bb51-427e11844020 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:15.209339 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e920cd1-eacd-453c-9399-dbd0ac67e678 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029447e782484e8f8a704b683ede1819', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:15.209976 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e920cd1-eacd-453c-9399-dbd0ac67e678 service glance] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:15.210250 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e920cd1-eacd-453c-9399-dbd0ac67e678 service glance] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:15.210516 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e920cd1-eacd-453c-9399-dbd0ac67e678 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:15.218664 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e920cd1-eacd-453c-9399-dbd0ac67e678 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:15.220067 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e920cd1-eacd-453c-9399-dbd0ac67e678 service glance] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:15.228658 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1469/2905] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:25.133186 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:25.224818 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:48.414688 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be5a59-778e-4c00-8726-7b22f3a571f3 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:48.475250 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be5a59-778e-4c00-8726-7b22f3a571f3 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:48.479724 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:48.480340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:48.480615 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:48.480884 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] PATH_INFO: `/v3/users/646bf439f58145c28acc5599057d51f3` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:48.482039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=646bf439f58145c28acc5599057d51f3)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:48.484554 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:48.528921 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646bf439f58145c28acc5599057d51f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:48.530122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646bf439f58145c28acc5599057d51f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:48.537231 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646bf439f58145c28acc5599057d51f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:48.549617 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646bf439f58145c28acc5599057d51f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:48.556957 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646bf439f58145c28acc5599057d51f3'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:48.562774 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1437/2906] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:05:48 2022] DELETE /identity/v3/users/646bf439f58145c28acc5599057d51f3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:48.567211 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec680d7-7f02-4be5-9f5e-9550ccfea425 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:48.567730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec680d7-7f02-4be5-9f5e-9550ccfea425 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:48.567730 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec680d7-7f02-4be5-9f5e-9550ccfea425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:48.586382 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-aec680d7-7f02-4be5-9f5e-9550ccfea425 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:48.587574 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-aec680d7-7f02-4be5-9f5e-9550ccfea425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:48.645022 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1470/2907] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:48.657545 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:48.659034 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:48.717116 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48f152-4172-4982-9673-8d648f87dfb1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:48.720721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a9bb1-edf1-472c-b2ce-8f7204e4f707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:48.721349 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9bb1-edf1-472c-b2ce-8f7204e4f707 service neutron] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:48.721627 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9bb1-edf1-472c-b2ce-8f7204e4f707 service neutron] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:48.721899 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9bb1-edf1-472c-b2ce-8f7204e4f707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:48.730650 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a9bb1-edf1-472c-b2ce-8f7204e4f707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:48.732062 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a9bb1-edf1-472c-b2ce-8f7204e4f707 service neutron] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:48.741665 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1438/2908] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 02:05:49.010427 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec680d7-7f02-4be5-9f5e-9550ccfea425 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:49.012545 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-aec680d7-7f02-4be5-9f5e-9550ccfea425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:49.033530 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec680d7-7f02-4be5-9f5e-9550ccfea425 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:49.038439 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:49.039201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.047825 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.048120 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] PATH_INFO: `/v3/projects/905d7c2123df411488e6a86e40a5b4a1` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.049414 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=905d7c2123df411488e6a86e40a5b4a1)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:49.051379 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:49.100704 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '905d7c2123df411488e6a86e40a5b4a1', 'request_id': 'req-62cc3e89-94f3-4b77-984d-0074f7f7bc38'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:49.107317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '905d7c2123df411488e6a86e40a5b4a1', 'request_id': 'req-62cc3e89-94f3-4b77-984d-0074f7f7bc38'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:49.113242 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '905d7c2123df411488e6a86e40a5b4a1', 'request_id': 'req-62cc3e89-94f3-4b77-984d-0074f7f7bc38'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:49.127176 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] The token cache is being invalidate because project 905d7c2123df411488e6a86e40a5b4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87883) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:05:49.127463 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:49.128079 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-62cc3e89-94f3-4b77-984d-0074f7f7bc38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:05:49.129617 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1471/2909] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:05:49 2022] DELETE /identity/v3/projects/905d7c2123df411488e6a86e40a5b4a1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:49.134494 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361dfa-cb18-4676-879d-1b18ecf1ce6b None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.134769 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361dfa-cb18-4676-879d-1b18ecf1ce6b None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.135055 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f361dfa-cb18-4676-879d-1b18ecf1ce6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.154621 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-6f361dfa-cb18-4676-879d-1b18ecf1ce6b None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:49.155757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-6f361dfa-cb18-4676-879d-1b18ecf1ce6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:49.172391 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:49.214308 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1439/2910] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:49.223139 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aee90-e0fc-42d6-89e8-9d2511f6e8df None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.223423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aee90-e0fc-42d6-89e8-9d2511f6e8df None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.223720 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4aee90-e0fc-42d6-89e8-9d2511f6e8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.243788 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-ed4aee90-e0fc-42d6-89e8-9d2511f6e8df None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:49.244970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-ed4aee90-e0fc-42d6-89e8-9d2511f6e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:49.253475 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1472/2911] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:49.257467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361dfa-cb18-4676-879d-1b18ecf1ce6b None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:49.266851 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f361dfa-cb18-4676-879d-1b18ecf1ce6b None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:49.271320 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:49.271904 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.272177 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.272446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] PATH_INFO: `/v3/domains` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.273103 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87884) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 02:05:49.273404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:49.274966 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:49.286172 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1440/2912] 10.222.0.15 () {58 vars in 1153 bytes} [Fri Sep 9 02:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 02:05:49.290364 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aee90-e0fc-42d6-89e8-9d2511f6e8df None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:49.300037 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4aee90-e0fc-42d6-89e8-9d2511f6e8df None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:49.304599 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278642a-cce9-4e23-8186-055bb3f70957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:49.305225 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278642a-cce9-4e23-8186-055bb3f70957 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.305502 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278642a-cce9-4e23-8186-055bb3f70957 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.305790 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278642a-cce9-4e23-8186-055bb3f70957 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.306406 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278642a-cce9-4e23-8186-055bb3f70957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:49.308270 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278642a-cce9-4e23-8186-055bb3f70957 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:49.334970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1473/2913] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:49 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:49.339122 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:49.349142 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50bc41-cfa2-4946-882c-9652cc21ba7e admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:49.405768 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7805d-5ade-4910-bda9-1e127f60142f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:49.406291 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7805d-5ade-4910-bda9-1e127f60142f admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.406563 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7805d-5ade-4910-bda9-1e127f60142f admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.406829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7805d-5ade-4910-bda9-1e127f60142f admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.407457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7805d-5ade-4910-bda9-1e127f60142f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:49.409371 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7805d-5ade-4910-bda9-1e127f60142f admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:49.430746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1441/2914] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:49 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:49.434793 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278642a-cce9-4e23-8186-055bb3f70957 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:49.444838 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278642a-cce9-4e23-8186-055bb3f70957 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:49.449681 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d04465-54d2-4b75-b8a4-e2d3e5d79468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:49.450277 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04465-54d2-4b75-b8a4-e2d3e5d79468 admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.450555 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04465-54d2-4b75-b8a4-e2d3e5d79468 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.450830 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d04465-54d2-4b75-b8a4-e2d3e5d79468 admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.451465 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d04465-54d2-4b75-b8a4-e2d3e5d79468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:49.453641 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d04465-54d2-4b75-b8a4-e2d3e5d79468 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:49.463462 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1474/2915] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:49.467189 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7805d-5ade-4910-bda9-1e127f60142f admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:49.477078 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7805d-5ade-4910-bda9-1e127f60142f admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:49.481633 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aec6cb-528c-4527-9235-ae6d60b76ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:49.482216 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec6cb-528c-4527-9235-ae6d60b76ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.482497 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec6cb-528c-4527-9235-ae6d60b76ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.482793 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec6cb-528c-4527-9235-ae6d60b76ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.483424 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aec6cb-528c-4527-9235-ae6d60b76ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:49.485780 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aec6cb-528c-4527-9235-ae6d60b76ce4 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:49.496083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1442/2916] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:49.500732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d04465-54d2-4b75-b8a4-e2d3e5d79468 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:49.510541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d04465-54d2-4b75-b8a4-e2d3e5d79468 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:49.515091 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:49.515718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.515991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.516260 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] PATH_INFO: `/v3/projects/eab48d377890454abfabbfa3053eaa57/users/4d5be7be998f438d8ecb854412d7dbf5/roles/18bebd2151824cdfa386be482ff8114b` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.530223 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab48d377890454abfabbfa3053eaa57, user_id=4d5be7be998f438d8ecb854412d7dbf5, role_id=18bebd2151824cdfa386be482ff8114b)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:49.533345 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:49.545769 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1475/2917] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:49 2022] PUT /identity/v3/projects/eab48d377890454abfabbfa3053eaa57/users/4d5be7be998f438d8ecb854412d7dbf5/roles/18bebd2151824cdfa386be482ff8114b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:05:49.551157 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686974-3c31-4cf8-9d11-62cf11702ca2 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.551439 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686974-3c31-4cf8-9d11-62cf11702ca2 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.551711 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac686974-3c31-4cf8-9d11-62cf11702ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.570227 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-ac686974-3c31-4cf8-9d11-62cf11702ca2 None None] MFA Rules not processed for user `2a2e93b317db4de6850826f625ccc3db`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:49.571411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ac686974-3c31-4cf8-9d11-62cf11702ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:49.628695 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1443/2918] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:49.638991 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:49.640826 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:49.664162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:49.701934 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b88c4-30a4-4171-a23f-d264f593e9f1 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:49.705707 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6fa63-23cf-4db4-b385-13139b27e6cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:49.706317 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6fa63-23cf-4db4-b385-13139b27e6cc service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:49.706598 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6fa63-23cf-4db4-b385-13139b27e6cc service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:49.706870 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f6fa63-23cf-4db4-b385-13139b27e6cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:49.715890 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6fa63-23cf-4db4-b385-13139b27e6cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:49.717340 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f6fa63-23cf-4db4-b385-13139b27e6cc service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:49.727136 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1476/2919] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:53.625869 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92550b82-eb66-4235-8dc8-6055a0d86a36 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:53.626446 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92550b82-eb66-4235-8dc8-6055a0d86a36 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:53.626571 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92550b82-eb66-4235-8dc8-6055a0d86a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:53.671525 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-92550b82-eb66-4235-8dc8-6055a0d86a36 None None] MFA Rules not processed for user `4d5be7be998f438d8ecb854412d7dbf5`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:53.672992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-92550b82-eb66-4235-8dc8-6055a0d86a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:53.697928 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:53.756995 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1444/2920] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:53.758174 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6fa63-23cf-4db4-b385-13139b27e6cc service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:53.769180 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f6fa63-23cf-4db4-b385-13139b27e6cc service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:53.774799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33627-a4e2-47d4-9101-f1b2447632b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:53.775402 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33627-a4e2-47d4-9101-f1b2447632b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:53.775679 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33627-a4e2-47d4-9101-f1b2447632b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:53.775948 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33627-a4e2-47d4-9101-f1b2447632b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:53.776576 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33627-a4e2-47d4-9101-f1b2447632b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:53.778532 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33627-a4e2-47d4-9101-f1b2447632b2 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:53.811210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1477/2921] 10.222.0.15 () {60 vars in 1153 bytes} [Fri Sep 9 02:05:53 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 02:05:53.815661 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92550b82-eb66-4235-8dc8-6055a0d86a36 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:53.827825 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92550b82-eb66-4235-8dc8-6055a0d86a36 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:53.890606 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372d0fb-e3e5-4ab2-a588-08d65aeddfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:53.890864 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372d0fb-e3e5-4ab2-a588-08d65aeddfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:53.891071 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372d0fb-e3e5-4ab2-a588-08d65aeddfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:53.891467 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372d0fb-e3e5-4ab2-a588-08d65aeddfe5 admin admin] PATH_INFO: `/v3/users` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:53.892145 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372d0fb-e3e5-4ab2-a588-08d65aeddfe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:53.894128 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372d0fb-e3e5-4ab2-a588-08d65aeddfe5 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:53.921541 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1445/2922] 10.222.0.15 () {60 vars in 1144 bytes} [Fri Sep 9 02:05:53 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 02:05:53.925943 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33627-a4e2-47d4-9101-f1b2447632b2 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:53.937524 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33627-a4e2-47d4-9101-f1b2447632b2 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:53.942808 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ecf1c-5ea2-4e02-8cc9-15f465ef891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:53.943677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ecf1c-5ea2-4e02-8cc9-15f465ef891b admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:53.944040 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ecf1c-5ea2-4e02-8cc9-15f465ef891b admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:53.944409 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ecf1c-5ea2-4e02-8cc9-15f465ef891b admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:53.945137 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ecf1c-5ea2-4e02-8cc9-15f465ef891b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:53.946805 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ecf1c-5ea2-4e02-8cc9-15f465ef891b admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:53.962015 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1478/2923] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:53.966017 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372d0fb-e3e5-4ab2-a588-08d65aeddfe5 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:53.976679 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372d0fb-e3e5-4ab2-a588-08d65aeddfe5 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:53.981417 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548eda6-a0a3-4a85-9cbb-593b69e8a9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:53.982057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548eda6-a0a3-4a85-9cbb-593b69e8a9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:53.982336 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548eda6-a0a3-4a85-9cbb-593b69e8a9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:53.982609 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548eda6-a0a3-4a85-9cbb-593b69e8a9a9 admin admin] PATH_INFO: `/v3/projects/031e23e8f201499e9caf6bf95923bf72/users/f22b257a3e544516800862a6c4c91b11/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:53.998347 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548eda6-a0a3-4a85-9cbb-593b69e8a9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031e23e8f201499e9caf6bf95923bf72, user_id=f22b257a3e544516800862a6c4c91b11, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:54.001601 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548eda6-a0a3-4a85-9cbb-593b69e8a9a9 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:54.016057 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1446/2924] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:53 2022] PUT /identity/v3/projects/031e23e8f201499e9caf6bf95923bf72/users/f22b257a3e544516800862a6c4c91b11/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:05:54.020061 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ecf1c-5ea2-4e02-8cc9-15f465ef891b admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:54.031336 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ecf1c-5ea2-4e02-8cc9-15f465ef891b admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:54.097241 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a0b7c-98ed-400b-8ea8-12ffc4a5b09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:54.098147 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0b7c-98ed-400b-8ea8-12ffc4a5b09a admin admin] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:54.098422 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0b7c-98ed-400b-8ea8-12ffc4a5b09a admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:54.098697 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0b7c-98ed-400b-8ea8-12ffc4a5b09a admin admin] PATH_INFO: `/v3/roles` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:54.099332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a0b7c-98ed-400b-8ea8-12ffc4a5b09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:54.100970 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a0b7c-98ed-400b-8ea8-12ffc4a5b09a admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:54.111654 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1479/2925] 10.222.0.15 () {58 vars in 1122 bytes} [Fri Sep 9 02:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 02:05:54.115745 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548eda6-a0a3-4a85-9cbb-593b69e8a9a9 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:54.126630 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548eda6-a0a3-4a85-9cbb-593b69e8a9a9 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:54.131505 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:54.132163 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:54.132443 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:54.132746 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] PATH_INFO: `/v3/projects/031e23e8f201499e9caf6bf95923bf72/users/f22b257a3e544516800862a6c4c91b11/roles/b131b61299154544a627315649090a49` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:54.134725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031e23e8f201499e9caf6bf95923bf72, user_id=f22b257a3e544516800862a6c4c91b11, role_id=b131b61299154544a627315649090a49)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:54.137411 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:54.150022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1447/2926] 10.222.0.15 () {60 vars in 1483 bytes} [Fri Sep 9 02:05:54 2022] PUT /identity/v3/projects/031e23e8f201499e9caf6bf95923bf72/users/f22b257a3e544516800862a6c4c91b11/roles/b131b61299154544a627315649090a49 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:05:58.774804 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e0489-2fac-4f97-9880-8f68d42e6874 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:58.775643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e0489-2fac-4f97-9880-8f68d42e6874 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:58.775643 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e0489-2fac-4f97-9880-8f68d42e6874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:58.805866 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-4e9e0489-2fac-4f97-9880-8f68d42e6874 None None] MFA Rules not processed for user `f22b257a3e544516800862a6c4c91b11`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:58.806998 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-4e9e0489-2fac-4f97-9880-8f68d42e6874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:58.887566 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1480/2927] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:05:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:58.900331 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:58.902083 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:58.964551 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afebf1c-5461-4547-b520-f4e404eb1dc1 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:58.968678 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029447e782484e8f8a704b683ede1819', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:58.969314 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:58.969587 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:58.969858 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:58.996311 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:59.032340 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:59.033922 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:59.043716 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1448/2928] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:05:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:05:59.182968 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64ae98-99d3-48cc-a73f-01d187d2a968 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:59.183256 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64ae98-99d3-48cc-a73f-01d187d2a968 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:59.183557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64ae98-99d3-48cc-a73f-01d187d2a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:59.203901 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-5c64ae98-99d3-48cc-a73f-01d187d2a968 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:05:59.205145 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-5c64ae98-99d3-48cc-a73f-01d187d2a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:59.210837 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:05:59.264195 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1481/2929] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 02:05:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:05:59.310823 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:05:59.312753 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:05:59.382408 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248010e-ca3d-40e8-a721-4331cfcbcca1 service glance] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:05:59.387243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:05:59.387837 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:05:59.388111 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:05:59.388377 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:05:59.396788 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:05:59.398367 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:05:59.407161 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1449/2930] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 02:05:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 02:06:07.955665 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6d64-1396-4c40-89b1-4fb802d03021 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:06:07.956185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6d64-1396-4c40-89b1-4fb802d03021 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:06:07.956185 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6d64-1396-4c40-89b1-4fb802d03021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:06:07.974744 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-414a6d64-1396-4c40-89b1-4fb802d03021 None None] MFA Rules not processed for user `4d5be7be998f438d8ecb854412d7dbf5`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:06:07.975926 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-414a6d64-1396-4c40-89b1-4fb802d03021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:07.983262 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1482/2931] 10.222.0.15 () {58 vars in 958 bytes} [Fri Sep 9 02:06:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:06:09.403723 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=48 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:06:09.581441 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:06:09.583119 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:09.653591 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233c457-6d58-406a-aeb4-7d5103f3e3a8 service swift] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:06:09.658775 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:06:09.659317 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:06:09.659596 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:06:09.659883 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:06:09.661620 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:09.673353 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:06:09.674903 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:06:09.684725 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1450/2932] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:06:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:06:11.208708 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2729-54b9-4ebb-badd-bb2c4d43ae62 None None] REQUEST_METHOD: `POST` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:06:11.209237 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2729-54b9-4ebb-badd-bb2c4d43ae62 None None] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:06:11.209359 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff2729-54b9-4ebb-badd-bb2c4d43ae62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:06:11.228162 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.auth.core [None req-c7ff2729-54b9-4ebb-badd-bb2c4d43ae62 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87883) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:06:11.229393 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c7ff2729-54b9-4ebb-badd-bb2c4d43ae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:11.237282 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1483/2933] 10.222.0.15 () {58 vars in 1009 bytes} [Fri Sep 9 02:06:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:06:11.250273 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:06:11.262593 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f6f34-9bda-403e-a07d-dec58a7c35d4 service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:06:11.267613 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c28d2-5bc9-44c4-80d6-db548118c274 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:06:11.268206 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c28d2-5bc9-44c4-80d6-db548118c274 service swift] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:06:11.268480 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c28d2-5bc9-44c4-80d6-db548118c274 service swift] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:06:11.268748 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c28d2-5bc9-44c4-80d6-db548118c274 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:06:11.276740 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c28d2-5bc9-44c4-80d6-db548118c274 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:06:11.278685 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c28d2-5bc9-44c4-80d6-db548118c274 service swift] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:06:11.287806 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1451/2934] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 02:06:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 02:06:21.222557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=26 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:06:21.284569 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=38 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:06:22.313161 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff2729-54b9-4ebb-badd-bb2c4d43ae62 None None] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:06:22.314677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-c7ff2729-54b9-4ebb-badd-bb2c4d43ae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:22.392233 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff2729-54b9-4ebb-badd-bb2c4d43ae62 None None] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:06:22.397843 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:06:22.398457 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:06:22.398732 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:06:22.399003 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:06:22.400718 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:22.414335 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:06:22.416004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:06:22.427131 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1484/2935] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:06:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:06:28.788597 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c28d2-5bc9-44c4-80d6-db548118c274 service swift] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:06:28.831992 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c28d2-5bc9-44c4-80d6-db548118c274 service swift] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:06:28.838137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:06:28.838743 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:06:28.839025 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:06:28.839297 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:06:28.841190 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:28.875437 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:06:28.877002 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:06:28.887326 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1452/2936] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:06:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:06:32.422964 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=34 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:06:38.882845 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=28 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:06:40.971993 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:06:40.973426 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:41.056331 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8172021-20ea-47d7-b136-72d34c155b48 service nova] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:06:41.059982 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d607-98f2-4ff8-ab35-dd8c58b18aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:06:41.060577 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d607-98f2-4ff8-ab35-dd8c58b18aed service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:06:41.060872 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d607-98f2-4ff8-ab35-dd8c58b18aed service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:06:41.061184 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d607-98f2-4ff8-ab35-dd8c58b18aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:06:41.070887 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97d607-98f2-4ff8-ab35-dd8c58b18aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:06:41.072656 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97d607-98f2-4ff8-ab35-dd8c58b18aed service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:06:41.083773 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1485/2937] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:06:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:06:51.080013 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=39 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:06:51.957854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:06:51.978033 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126c9ce-39bf-4320-a589-805ba67aad4e service nova] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:06:51.983272 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:06:51.983897 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:06:51.984173 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:06:51.984442 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:06:51.986450 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:06:52.010557 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:06:52.012124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:06:52.023114 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1453/2938] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:06:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:07:02.018555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=15 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:09:01.055163 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d607-98f2-4ff8-ab35-dd8c58b18aed service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:01.073541 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d607-98f2-4ff8-ab35-dd8c58b18aed service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:01.077143 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341277e7e3144e5ea93fa657916821e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:01.077729 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:01.078004 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:01.078272 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:01.079972 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:01.112352 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:01.113789 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:01.136093 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1486/2939] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:09:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:09:02.629039 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:02.630698 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:02.647577 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c391-bb14-405c-adc6-7bc8130ac41b service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:02.653889 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b49e2-0522-46f7-9ec0-a40c8840935c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63d93adb4e4044ae79fb5aa7d8f5e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:02.654660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b49e2-0522-46f7-9ec0-a40c8840935c service nova] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:02.655084 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b49e2-0522-46f7-9ec0-a40c8840935c service nova] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:02.655461 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b49e2-0522-46f7-9ec0-a40c8840935c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:02.657508 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-ff2b49e2-0522-46f7-9ec0-a40c8840935c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:02.672022 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b49e2-0522-46f7-9ec0-a40c8840935c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:02.673798 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b49e2-0522-46f7-9ec0-a40c8840935c service nova] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:02.686410 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1454/2940] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:09:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:09:03.992043 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:03.993445 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:04.074867 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e8bfb-fcaf-4f61-99d3-55fe9b2c70ab service neutron] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:04.079008 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df27a2ce80b94237bd2b9ecfe35109cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:04.079608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:04.079883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:04.080150 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:04.081936 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:04.094906 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:04.096303 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:04.107534 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1487/2941] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:09:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:09:10.092701 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ad07d-eaa2-4321-bb23-58310099cbc3 None None] REQUEST_METHOD: `POST` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:10.093167 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ad07d-eaa2-4321-bb23-58310099cbc3 None None] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:10.093300 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ad07d-eaa2-4321-bb23-58310099cbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:10.122660 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.auth.core [None req-182ad07d-eaa2-4321-bb23-58310099cbc3 None None] MFA Rules not processed for user `9cfe3dd0af154707bdaae5b52cbd6539`. Rule list: `[]` (Enabled: `True`). {{(pid=87884) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 02:09:10.123854 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-182ad07d-eaa2-4321-bb23-58310099cbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:10.130809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1455/2942] 10.222.0.15 () {58 vars in 967 bytes} [Fri Sep 9 02:09:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 02:09:10.175578 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:10.186754 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafbb1b-3c20-4440-8e63-12b4ae898109 service cinder] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:10.191742 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5bdea-2403-4ad5-9311-27e4f1ec3e85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0d08965bc9c4ca29812d304bfb4ab4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:10.192332 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5bdea-2403-4ad5-9311-27e4f1ec3e85 service swift] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:10.192608 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5bdea-2403-4ad5-9311-27e4f1ec3e85 service swift] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:10.192880 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5bdea-2403-4ad5-9311-27e4f1ec3e85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:10.201275 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5bdea-2403-4ad5-9311-27e4f1ec3e85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:10.202817 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5bdea-2403-4ad5-9311-27e4f1ec3e85 service swift] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:10.210639 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1488/2943] 10.222.0.15 () {60 vars in 1384 bytes} [Fri Sep 9 02:09:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 02:09:13.573542 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ad07d-eaa2-4321-bb23-58310099cbc3 None None] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:13.589860 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ad07d-eaa2-4321-bb23-58310099cbc3 None None] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:13.596923 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:13.597721 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] REQUEST_METHOD: `GET` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:13.598137 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:13.598533 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:13.600841 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:13.614073 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:13.615757 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:13.629388 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1456/2944] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:09:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 02:09:16.508204 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5bdea-2403-4ad5-9311-27e4f1ec3e85 service swift] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:16.590468 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5bdea-2403-4ad5-9311-27e4f1ec3e85 service swift] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:16.595011 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:16.595627 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:16.595900 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:16.596167 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] PATH_INFO: `/v3/users/4d5be7be998f438d8ecb854412d7dbf5` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:16.597292 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5be7be998f438d8ecb854412d7dbf5)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:16.599189 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:16.604255 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=100 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:09:16.650289 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5be7be998f438d8ecb854412d7dbf5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:16.651435 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5be7be998f438d8ecb854412d7dbf5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:16.663677 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5be7be998f438d8ecb854412d7dbf5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:16.671595 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5be7be998f438d8ecb854412d7dbf5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:16.672203 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5be7be998f438d8ecb854412d7dbf5'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:16.684497 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1489/2945] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:09:16 2022] DELETE /identity/v3/users/4d5be7be998f438d8ecb854412d7dbf5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:09:16.854321 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:16.865581 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd09eb-bed4-4ebb-8ddf-872a44754722 service placement] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:16.928908 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:16.929575 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:16.929856 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:16.930124 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] PATH_INFO: `/v3/projects/eab48d377890454abfabbfa3053eaa57` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:16.931255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab48d377890454abfabbfa3053eaa57)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:16.933196 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:16.986335 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab48d377890454abfabbfa3053eaa57', 'request_id': 'req-3a89f709-c941-4bee-bd4a-93e009cfcf80'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:16.991770 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab48d377890454abfabbfa3053eaa57', 'request_id': 'req-3a89f709-c941-4bee-bd4a-93e009cfcf80'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:16.998306 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab48d377890454abfabbfa3053eaa57', 'request_id': 'req-3a89f709-c941-4bee-bd4a-93e009cfcf80'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.023880 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] The token cache is being invalidate because project eab48d377890454abfabbfa3053eaa57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:09:17.024149 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.024809 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.026933 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1457/2946] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:09:16 2022] DELETE /identity/v3/projects/eab48d377890454abfabbfa3053eaa57 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:09:17.031054 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:17.032431 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:17.045210 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859037ec-5486-42a0-934a-12eb9f873d38 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:17.099913 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:17.100526 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:17.100799 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:17.101082 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] PATH_INFO: `/v3/users/f22b257a3e544516800862a6c4c91b11` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:17.102201 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f22b257a3e544516800862a6c4c91b11)` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:17.104134 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:17.149883 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f22b257a3e544516800862a6c4c91b11'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.150600 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f22b257a3e544516800862a6c4c91b11'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.162474 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f22b257a3e544516800862a6c4c91b11'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.169612 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f22b257a3e544516800862a6c4c91b11'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.170222 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.notifications [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f22b257a3e544516800862a6c4c91b11'} {{(pid=87883) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.182767 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1490/2947] 10.222.0.15 () {58 vars in 1224 bytes} [Fri Sep 9 02:09:17 2022] DELETE /identity/v3/users/f22b257a3e544516800862a6c4c91b11 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 02:09:17.360942 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] Authenticating user token {{(pid=87884) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:09:17.362559 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.fernet_utils [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87884) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:09:17.376144 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89f709-c941-4bee-bd4a-93e009cfcf80 admin admin] Validating token access rules against request {{(pid=87884) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:09:17.409255 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:09:17.433240 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a2e93b317db4de6850826f625ccc3db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af9d764870ec46c58ab37a817dc46423', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87884) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:09:17.433872 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:09:17.434150 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:09:17.434423 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] PATH_INFO: `/v3/projects/031e23e8f201499e9caf6bf95923bf72` {{(pid=87884) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:09:17.435555 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031e23e8f201499e9caf6bf95923bf72)` {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:09:17.437515 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] RBAC: Authorization granted {{(pid=87884) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:09:17.481829 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031e23e8f201499e9caf6bf95923bf72', 'request_id': 'req-92faabcc-ed0b-4923-9827-9eafc160e4d8'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.487457 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031e23e8f201499e9caf6bf95923bf72', 'request_id': 'req-92faabcc-ed0b-4923-9827-9eafc160e4d8'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.493404 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031e23e8f201499e9caf6bf95923bf72', 'request_id': 'req-92faabcc-ed0b-4923-9827-9eafc160e4d8'} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.504847 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] The token cache is being invalidate because project 031e23e8f201499e9caf6bf95923bf72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87884) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 02:09:17.505165 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.505783 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG keystone.notifications [None req-92faabcc-ed0b-4923-9827-9eafc160e4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87884) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 02:09:17.507243 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: [pid: 87884|app: 0|req: 1458/2948] 10.222.0.15 () {58 vars in 1233 bytes} [Fri Sep 9 02:09:17 2022] DELETE /identity/v3/projects/031e23e8f201499e9caf6bf95923bf72 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 02:09:27.178933 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=77,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:09:27.502072 dv-cinder-852210-12-iscsi devstack@keystone.service[87884]: DEBUG dbcounter [-] [87884] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87884) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 02:11:55.050800 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] Authenticating user token {{(pid=87883) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 02:11:55.052226 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:11:55.119548 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f9809-b88f-416a-a80f-12079b732930 admin admin] Validating token access rules against request {{(pid=87883) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 02:11:55.178666 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396af148-7f8d-4629-b4b0-075460ffe2bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '365e1f44e9d849118e64975b4c3d674b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '819c1fdfa2a8401f960f551c31cb82e6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87883) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 02:11:55.179276 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396af148-7f8d-4629-b4b0-075460ffe2bd service placement] REQUEST_METHOD: `GET` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 02:11:55.179557 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396af148-7f8d-4629-b4b0-075460ffe2bd service placement] SCRIPT_NAME: `/identity` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 02:11:55.179875 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396af148-7f8d-4629-b4b0-075460ffe2bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87883) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 02:11:55.181668 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.fernet_utils [None req-396af148-7f8d-4629-b4b0-075460ffe2bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87883) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 02:11:55.193284 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396af148-7f8d-4629-b4b0-075460ffe2bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 02:11:55.194836 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396af148-7f8d-4629-b4b0-075460ffe2bd service placement] RBAC: Authorization granted {{(pid=87883) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 02:11:55.204614 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: [pid: 87883|app: 0|req: 1491/2949] 10.222.0.15 () {60 vars in 1365 bytes} [Fri Sep 9 02:11:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 02:12:05.200423 dv-cinder-852210-12-iscsi devstack@keystone.service[87883]: DEBUG dbcounter [-] [87883] Writing DB stats keystone:SELECT=58 {{(pid=87883) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}